notonthehighstreet.com is the UK’s largest curated online marketplace that brings together over 5,000 of the UK’s best small creative businesses offering over 170,000 original products, many of which are bespoke or built to order. The primary website www.notonthehighstreet.com receives millions of hits everyday, and the traffic increases by an order of magnitude in the months before Christmas.
notonthehighstreet.com has grown rapidly in the nine years since it was founded, and with the forecast of continued growth the senior management team sought to establish a technology platform that would support several initiatives. They included the predicted increase in customer traffic, entry into new markets, and new functionality required to drive business aspirations. As the primary notonthehighstreet.com application had grown in functionality and complexity within a relatively short space of time, the notonthehighstreet.com leadership team were keen to assess the future capabilities of this platform, both in terms of architectural quality and also operational scalability. In particular, meeting traffic needs in the final quarter of the year, such as spiky traffic patterns around ‘Black Friday’ and ‘Cyber Monday’ were of vital importance to the business.
notonthehighstreet.com had previously invested in establishing an effective agile development methodology, and had begun embracing the ‘DevOps’ philosophy, which meant the technical team was capable of deploying changes to an application running in a production environment within minutes. The technology leadership were keen to build upon this while also refreshing the underlying infrastructure to enable more rapid scaling, a programming language neutral deployment mechanism, and a developer-friendly approach to creating and maintaining programmable infrastructure.
“Continuous Integration and Deployment allows us to reduce cost and risk through incremental updates to our applications. We get feedback faster and this better equips us to make decisions on future direction. ”
The leadership were also keen to increase diversity within the technology stack, for example adding a capability around the JVM language platform. There was also interest throughout the team to evaluate the potential for service-oriented architecture (SOA) and microservices.
With the primary goals of moving to a service-oriented architecture, improving operational scalability and increasing diversity of programming languages within the application stack the notonthehighstreet.com management team sought the advice of OpenCredo. Initially a small proof-of-concept Java order management service was created by OpenCredo consultants pair programming with NOTHS permanent staff. An important part of this engagement was knowledge sharing and teaching of architectural best practices, of which the NOTHS team were very keen to learn. The success of this initial engagement lead to a further six month engagement with a small group of OpenCredo consultants being placed within and working directly with the NOTHS technical team.
This six month engagement focused primarily on introducing a service-oriented architecture into the NOTHS application stack, pioneering new development techniques and patterns, and also building a permanent Java development team. A series of hands-on workshops and ‘brown bag’ knowledge sessions were run by the OpenCredo team, and a collection of new Java-based services were created in combination with the newly formed Java team. New development techniques were also introduced, including behaviour-driven design (BDD), the notion of creating ‘anti-fragile’ (distributed and cloud native) software, and new design paradigms were also demonstrated, such as event sourcing and CQRS. OpenCredo were instrumental in helping us broaden our tech stack. Java has not only enabled us to recruit from a much larger developer network but has added enterprise speed and stability.
Due to the success of the initial six month engagement, OpenCredo were invited to continue working with the NOTHS team on an additional six month engagement to complete the transformation into a leading-edge platform. This work focused primarily in the migration of the existing notonthehighstreet.com infrastructure to the AWS public cloud, the creation of additional services, and the continued increase in architectural awareness. Working closely with the NOTHS DevOps team, a new platform was designed, prototyped and implemented using technologies such as Docker, Apache Mesos, Mesosphere Marathon, Chronos, HashiCorp Consul and the AWS suite of products. All infrastructure was created programmatically using technologies such as Ansible, AWS CloudFormation and HashiCorp Packer. The cloud migration was completed in the summer of 2015, and both application response times and footprint were reduced. Initial load testing also indicated that the platform would be able to elastically scale to meet the requirements of the upcoming ‘Black Friday’
“Treating infrastructure as code has greatly reduced the time and effort spent on server provision and administration, it has also brought developer and operations teams closer together by implementing a common understanding.”
Although the primary transformational work is complete, notonthehighstreet.com and OpenCredo continue to enjoy a fruitful partnership, and members of the OpenCredo team are still working alongside the NOTHS technical team. A series of joint conference presentations and blog posts are also planned, where additional details of the work will be shared with the international software development community.
OpenCredo and the permanent notonthehighstreet.com technical team collaborated closely on the creation of a flexible service-oriented architecture that is packaged using leading-edge container technology and deployed onto automatically provisioned and dynamically scalable cloud-based infrastructure. OpenCredo also lead the recruitment and training of a new Java-based development team, which successfully allowed notonthehighstreet.com to diversify its use of technology throughout the product development teams.
The OpenCredo consultants worked closely with the NOTHS team, and demonstrated key architectural principles for creating modular, scalable, and distributed applications. Working closely alongside the notonthehighstreet.com DevOps team, leading-edge techniques such as programmable infrastructure, containerisation and cluster management were implemented, which radically improved the ability to scale to meet customer demands.
The OpenCredo consultants worked closely with the entire NOTHS technical and product owners team, and constantly sought to transfer both knowledge and skills. When the primary year-long engagement concluded all knowledge had been shared (and where appropriate transferred) to permanent NOTHS team members
notonthehighstreet.com – the UK’s largest curate online marketplace, which brings together over 5,000 of the UK’s best small creative businesses offering over 170,000 original products
Enable service-oriented architecture: Promoting independent deployment and scalability, and allow separation of service/functionality development lifecycles
Maintain, or reduce, application response times: For an e-commerce website like www.notonthehighstreet.com, delays in response time reduce revenue
Act as pioneers for introducing technologies and technique that will reduce development lead times and improve delivery quality, such as behaviour-driven design, code quality analysis and programmable cloud infrastructure
Enhance awareness and quality of software architecture throughout the entire notonthehighstreet.com application suite
Create, develop and train permanent team skilled in Java programming language
Develop skills to allow the creation and maintenance of a scalable platform that meets the demands of predicted customer traffic and business aspirations for the next five years
Java 8 and Spring Boot
Swagger and Pact (JVM and Ruby)
Jenkins, Sonar and Nexus
Apache Mesos, using Mesosphere Marathon for long-running services and Chronos for batch processing jobs
Amazon Web Services (AWS) EC2, RDS, S3, CloudFormation
HashiCorp Vagrant, Packer and Consul
notonthehighstreet.com and OpenCredo worked closely together to create a scalable distributed application platform that will enable and support the planned business growth over the next five years
OpenCredo successfully increased the diversity of application development options within the notonthehighstreet.com technical team by introducing the Java programming language (and associated JVM runtime stack), and were responsible for the recruitment and training of the permanent team members
OpenCredo provided architectural guidance and technical leadership throughout the transformation process, and lead training workshops, assisted with requirements gathering and project management, and delivered corresponding presentations of planning, status and achievements to all levels of the business, from the technical team to the C*O level
OpenCredo believes great IT consultancy is founded in deep and broad real-world experience. Our highly capable and experienced people provide applied knowledge of the latest technologies and organisational transformation methodologies. These capabilities stretch across the entire spectrum of the software development process, from architecture through software engineering to operations (and ‘DevOps’), and encompass transformational skills such C*O-level strategy creation, organisational design and change management.
OpenCredo are transparent, pragmatic and objective-driven, and we act as a trusted advisor to our clients. We are founded on excellence in software engineering, and appreciate that sustainable and lasting change often occurs when transformation is driven throughout an organisation, not just at the technology level. We deliver tangible value, not just slide decks or fluff.