MoveWORK and the challenges of massive data processing
March 20, 2017
« With 200 million calculations and 20 million requests per day, MoveWORK plays in the big leagues and develops in partnership with world number 1 AWS to tackle the new generation of BI from BigData and IoT. Performance has a price, because we are talking about hundreds of thousands of euros a year, but MoveWORK will stop at nothing to achieve its objectives and satisfy and retain its customers with an exceptional technical platform », says Patrick Tavaris, Group President.
MoveWORK, a European expert in business traceability and Business Intelligence, has been offering online solutions since 2009, enabling companies in the multi-service sector (reception, maintenance and security) to collect data directly in the field. Its flagship solution, DataFlow, provides managers with a 360 business view that enables continuous performance monitoring and improvement.
Luc BONNIN, Director of R&D programmes at MoveWORK, explains…
« Our expertise lies in our ability to analyse raw data, making it relevant and intelligible to the manager. This data is first collected by the employee (reception, maintenance or security agent, etc.), but also comes from a wide variety of sources: traffic sensors, live weather, data from satisfaction kiosks, information on vehicle traffic, etc. For Europe, this means hundreds of millions of pieces of data collected and processed every day in real time, which requires reliable servers and solutions. We found all this at Amazon Web Services (AWS). »
- Servers that support the variation in number of calls and data exchanged
- Collect and analyse data 24/7
- Over 20,000,000 database connections every day
Reliability at the heart of the service
« We had confidence in our products but not complete confidence in our suppliers. However, given our business, our requirements and the quality of service we want to offer our customers, it was essential we had reliable solutions and suppliers 24/7 », explains Luc Bonnin. «We detect the absences and anomalies as they occur. If a server is not working and we lose 2 minutes of time, it can be very serious. For example, you need to be able to know that an operating room has been cleaned or not, because if the person in charge doesn’t have the information quickly because of a down server, it can have dramatic consequences on the health of patients. »
In search of the ideal supplier, MoveWORK has worked with several partners since its creation. « To move forward, we had to get our hands dirty on the hardware part (PaaS) because we had semi-shared servers », says Luc Bonnin. « We had issues related to availability time, platform, structure and provider that were not 100% reliable. »
The second issue for MoveWORK was its development. « We designed our R&D program at the same time as the creation of the company, which in our case involves data, big data, data mining and artificial intelligence. To grow, move forward and innovate, we needed to collect a lot more data and more varied data, and we needed an ultra-robust and reliable PaaS platform. »
Adapted support from migration
To address this two-fold issue, MoveWORK migrated to AWS in August 2015. « When we knocked on the door of AWS, we didn’t expect support with such a high level of quality: we were “small” compared to other customers like Spotify or Netflix for example. But we got it! Of course, we have trained in AWS services, but we have received the help, support and advice we need » says Luc Bonnin.
« MoveWORK migrated about 50 servers and its central database in about twenty hours. This without losing a single piece of data, involving hundreds of billions of structured pieces of information and only cutting the servers out for about fifteen minutes. The migration went perfectly. For every question, every doubt, we had an answer. With 5 years of data history, we couldn’t make any mistakes! »
Focus on our core business: data analysis
« We were very concerned about data storage, the cornerstone to any IT company. This was the main reason that motivated us to switch to AWS. We moved to Amazon RDS with the MySQL engine and took the biggest server available in the Ireland zone, in multi A-Z so that the back-ups are automatically managed. Since our arrival at AWS, the database has never faltered. We have never experienced a service interruption. »
Being able to offload server management also liberated MoveWORK, which turned to the Amazon EC2 “package”. « Our external partners collect or exchange data using a MoveWORK web service. On average, 2,000,000 calls are made there during the day and at the time of migration, we even had peaks of 10,000,000,000 up to 20,000,000 calls in a few hours at night » recalls Luc Bonnin. « Thanks to AWS technologies and Elastic Load Balancing (ELB), the number of Amazon EC2 instances automatically evolves according to traffic, loads, processor, RAM… From 2-3 instances, we go to 5, 6 or even 10 if necessary. We never lose speed or power and we can process a huge amount of data. »
« We pay to use these servers, which allows us to automatically reduce the bill by 30% since we no longer spend time in configuration, installation, etc. Besides, with clients based in more than 60 countries around the world, having global coverage was essential. We couldn’t settle for just one server, no matter how powerful, in one region of the world. »
« Thanks to AWS, we have followed through on our ideas. We have multiplied by increased the number of pieces of data collected and analysed ten-fold, that is to say the intensity of what is required of the machines. At the same time, we gained 25% in speed, out of everything that can be calculated. Without the AWS cloud architecture that allows us to collect and analyse data, DataFlow would never have gone from the R&D lab to real development. We can now pursue our ambitions and boost our innovations using AWS solutions like Amazon Aurora or Amazon EMR… »