Sedex

The Solution

Our partnership with Sedex began with some Discovery work. This surfaced some specific use cases:

• A business needs to manage, store, analyse and disseminate huge volumes of data.
• The requirement for failure resilience.
• A desire to use open source technologies.
• The need to design and define a suitable data processing platform.

Working closely with the Sedex team, allowed us to outline a fit-for-purpose solution. This combined Neo4j, Google Compute Engine and Hortonworks Data Platform, a completely open-source technology. It was determined that Google Cloud Platform provided the most cost-effective and reliable option, with more broadband per pound. And Hortonworks made it easy to scale the data storage and computation needs, to address the increasing demands on Sedex’s system. Lastly, a pipeline was built using Hive. Data cubes from Hive queries would be stored in a PostgreSQL database for fast access by existing reporting technologies. 

During our four-month collaboration, we made use of agile working practices to iterate and deliver this solution:

‍

By working together, we were able to ensure a transfer of skills, empowering Sedex’s own people to operate their new platform independently.

‍

“We have the ability to produce large data sets very quickly and there are huge retailers in the U.S. who could very well become clients just because of that,” said José Copovi-King. “This is a huge step change for retailers and we’re now out in front of the competition in the CSR environment.”

“If a client has a specific requirement, we can rapidly meet that demand,” said José. “We had a request for a data security questionnaire and were able to carry this out very quickly by collecting data online. We can also design and sell our own reports – if we want to produce trend reports or benchmark companies against their peers we can now do so.”