The partnership began with a very lightweight Discovery phase.
Working collaboratively with National Journal, we established an understanding of typical working patterns for analysts, got into the detail of the data itself, as well as identified common queries and analysis techniques. Using our skills and experience in graph and broader data analysis techniques, we quickly arrived at the first version of the consolidated NSI graph data model which would accommodate the initial requirements. This provided the basis upon which further iterations could evolve.
Cognisant of the desire for a solution which was simple and not going to place a heavy operational burden on the engineering team needs to look after this, Google Cloud Platform (GCP) was collaboratively identified as the ideal basis to build upon. Moving into delivery, the broader workflow and ingestion solution started to take shape. The final ingestion workflow combined a number of GCP-managed services as well as some Python-based development. The end result was a solution where analysts upload data into the system, obtain initial validation and feedback, before ultimately landing up in Neo4j, having gone through a de-duplication process.
Having the end-to-end workflow in place, the opportunity to evolve the de-duplication and similarity detection logic was then tackled. Simple business rules, as well as more advanced algorithmic techniques, were used to deliver this aspect of the work. By continually working with the NSI analysts, the process, rules and algorithms were adapted and tuned to better identify and classify the data to achieve the most optimal outcome.
When we began our partnership with OpenCredo, we had done nearly 200 network research projects for 60+ clients and yet we had no way of tracking people, organizations, and the connections among them across all of that data.
OpenCredo helped us integrate that data, seamlessly flow it into our data visualization tool, and deal with a massive amount of data duplication issues. Through the process, we came to think about our data differently and have already begun to use this new frame to deliver excellence to our clients.
The experience of working with OpenCredo couldn’t have been better – they were highly professional, organized, and supremely competent in delivering this work to us.
– Luke Hartig, Executive Director
Technologies employed included Neo4j, Linkurious, Python and various GCP services.