INDUSTRY

Information technology

CHALLENGE:

An open source technology company that deals in real-time event reporting and logging,focusing specifically on error reporting.

TOP OUTCOMES:

Client's Company is an open source technology company that dealsin real-time event reporting and logging, focusing specifically on error reporting. They offer premium software development and strategy expertise to hundreds of companies worldwide.

Client provides real-time error reporting for JavaScript, Node, ASP.NET, Web API, WebForms, WPF, Console, and MVC apps.Everything is pulled into a dashboard that organizes each event into a simple yet robust and fully-featured view that gives you quick access to actionable data.

Client was using MongoDB for their primary document data store.They chose it for its ease of use, ability to scale out instead of up, and the cross platform story. It worked well until they started to grow creating problems scaling several features, such as real-time dashboards with automatic time zone shifting, backups, restores, and storage. Now they have seen great success migrating to Elasticsearch.

REAL-TIME DASHBOARDS:

Initially their real-time based dashboards with MongoDB required massive quantities of research on timezones, amonumental task on its own. They discovered a need to pre-aggregate data into 15 minute buckets of time (to account for different time zone offsets) into separate collections, in addition to our event collections. This allowed them to quickly return statistics data for any 15 minute time frame while adding the expense of additional storage and processing time needed to shift the buckets of stats into the desired time zone with each request.

BACKUP AND RESTORING:

A primary challenge before moving to Elasticsearch was backing up MongoDB. There is little documentation for doing so in production,which leads users to bring down the cluster to do a quick backup via MongoDump, zip the backup, and upload it to some destination. Though one of the most underrated features of using Elasticsearch is backups and restores! Our experience configuring and using this feature is efficient and enjoyable. We setup a Cron job on one of our nodes to run curator for cleanup, and to run a backup to azure blob storage via the cloud-azure plugin. The most useful feature is the ability to choose exactly what index to restore to a development machine if you wish to debug production data, while keeping your data secure and backed up in the cloud.

SCALING OUT:

There is nothing worse than waking up to find that a VM reboot of one of the nodes or adding a new node corrupted your MongoDB cluster configuration. This alarming situation can be resolved by restoring the cluster configuration. The MongoDB cluster configuration is something you don't touch if it's working.We love how easy it is to setup and manage an Elasticsearch cluster. No matter how you want it to run, it's just a simple configuration setting or REST call away! Most importantly, you can share your configuration with others to get feedback.

CONCLUSION:Client was pleased with our expert opinion to move to Elasticsearch and continue to receive additional value from it with every release.