Open Credo

80 items found: Search results for "needs" in all categories x

Programmable Infrastructure Needs Testing Too

March 20, 2017 | DevOps

Programmable Infrastructure Needs Testing Too

DevOps has swept the tech landscape. Now, many are discovering the benefits of programmable infrastructure. I have been lucky to work on many projects where we’ve taken advantage of tools such as Terraform, Ansible, or Chef.

 

Read More Read More

The Seven Heavenly Virtues a Project Manager needs

June 7, 2016 | Software Consultancy

The Seven Heavenly Virtues a Project Manager needs

In my previous blog post I explained to you the Seven Deadly Sins of Project Managers. With that post in mind, I would like to share with you my views on what a project manager needs to overcome these sins and become a valuable member of the team, and ultimately helping to achieve success on the project.

Read More Read More

Use Case: A Kafka Backup Solution

February 22, 2024 | Blog, Kafka

Use Case: A Kafka Backup Solution

Check out Peter Vegh’s latest blog where he explores a bespoke Kafka backup framework, discussing the approach, architecture and design process to implement and deliver the solution.

Read More Read More

How to use LLMs to Generate Coherent Long-Form Content using Hierarchical Expansion

October 30, 2023 | Blog

How to use LLMs to Generate Coherent Long-Form Content using Hierarchical Expansion

As impressive as they are, Large Language Models (LLMs) face difficulties when creating long-form content, primarily due to token limitations and inconsistencies in the output over time.

Together with Livy.ai, we developed a “Hierarchical Expansion” method to address these challenges and better the quality, flow, and structure of the content produced. Read further to learn more!

Read More Read More

Event Driven Load Testing

October 12, 2023 | Blog, Platform Engineering

Event Driven Load Testing

Check out our latest blog “Event Driven Load Testing” which explores how, through some smart automation techniques, testing strategies can be adapted to support scale-up organisations where there are potentially many disparate teams needing to work together.

Read More Read More

The Why’s, What’s and How’s of Kubernetes Operators

September 27, 2023 | Blog, Kubernetes

The Why’s, What’s and How’s of Kubernetes Operators

Learn to create your first Kubernetes operator by checking out our Senior Consultant Michal Tusnio’s latest blog, “Kubernetes Operators – Whys, Hows and Whats” where he takes you on a journey from zero to operator.

Read More Read More

Why should you upgrade your Terraform provider to the latest version of the API

August 17, 2023 | Blog, Terraform Provider

Why should you upgrade your Terraform provider to the latest version of the API

Check out John Sharpe and Will May’s latest blog where they give suggestions for Terraform Provider authors who are thinking about upgrading from SDKv2 to Framework

Read More Read More

Kafka: Navigating GDPR Compliance

June 8, 2023 | Blog, Kafka

Kafka: Navigating GDPR Compliance

Check out Greg Nuttall’s latest blog where he looks at the challenges posed by GDPR’s “Right to be Forgotten” in the context of Apache Kafka, and delves deeper into three strategies for overcoming them.

Read More Read More

TL;DR – Computational Governance in Data Mesh with OPA

April 21, 2023 | Blog, Data Engineering, Data Governance, Data Mesh, OPA

TL;DR – Computational Governance in Data Mesh with OPA

Check out Mateus Pimenta’s TL;DR video to learn how federated computational governance could be implemented using Open Policy Agent (OPA) and policy-as-code to support a successful Data Mesh architecture.

Read More Read More

Ingesting Big Data into Neo4j – Part 3

March 9, 2023 | Blog, Data Analysis, Neo4j

Ingesting Big Data into Neo4j – Part 3

Check out the last part of Ebru Cucen and Fahran Wallace’s blog series, in which they discuss their experience ingesting 400 million nodes and a billion relationships into Neo4j and what they discovered along the way.

Read More Read More

Controlling Kafka Data Flows using Open Policy Agent

August 2, 2022 | Blog, Kafka

Controlling Kafka Data Flows using Open Policy Agent

Read Matt Farrow’s blog as he explores the potential for using Open Policy Agent to filter and mask data being sent to and read from Apache Kafka.

Read More Read More

Exploring How Policy-as-Code and OPA Fit into the K8s World

November 4, 2021 | Kubernetes

Exploring How Policy-as-Code and OPA Fit into the K8s World

We always read that ‘security is everyone’s responsibility’. For any organisation, big or small, security should always be the primary concern—not a mere afterthought. In terms of Kubernetes, securing a cluster is challenging because it has so many moving parts and, apart from securing our Kubernetes environment, we also want to control what an end-user can do in our cluster.

To achieve these goals, we can start with the built-in features provided by Kubernetes like Role-Based Access Control (RBAC), Network Policies, Secrets Management, and Pod Security Policies (PSP). But we know these features are not enough. For example, we may want specific policies like ‘all pods must have specific labels’. And even if we have the policies in place, the next big question is how to enforce them on our Kubernetes cluster in an easy and repeatable manner.

In this blog post, we’ll address this challenge and other questions pertaining to OPA and how it can integrate into Kubernetes.

Read More Read More

Running the Cloud from your Kubernetes Cluster

September 2, 2021 | Blog, Cloud, Kubernetes

Running the Cloud from your Kubernetes Cluster

In this blog, Stuart compares the new approach of deploying cloud resources as Kubernetes custom resources rather than the (now) typical approach using Terraform – or cloud specific: CloudFormation (AWS), Deployment Manager (GCP). He also identifies what resources are suitable for this approach and which ones are not.

Read More Read More

Machine Learning at scale: first impressions of Kubeflow

April 20, 2021 | Data Engineering, Machine Learning, Software Consultancy

Machine Learning at scale: first impressions of Kubeflow

Our recent client was a Fintech who had ambitions to build a Machine Learning platform for real-time decision making. The client had significant Kubernetes proficiency, ran on the cloud, and had a strong preference for using free, open-source software over cloud-native offerings that come with lock-in. Several components were spiked with success (feature preparation with Apache Beam and Seldon for model serving performed particularly strongly). Kubeflow was one of the next technologies on our list of spikes, showing significant promise at the research stage and seemingly a good match for our client’s priorities and skills.

That platform slipped down the client’s priority list before completing the research for Kubeflow, so I wanted to see how that project might have turned out. Would Kubeflow have made the cut?

 

Read More Read More

Anthos – A Holistic Approach to your Hybrid Cloud initiative

February 17, 2021 | Blog, Cloud, Cloud Native, GCP, Open Source

Anthos – A Holistic Approach to your Hybrid Cloud initiative

Multi-cloud is rapidly becoming the cloud strategy of choice for enterprises looking to modernise their applications.

And the reason is simple – it gives them much more flexibility to host their workloads and data where it suits them best.

In this post, we focus on Google’s application modernisation solution Google Anthos and the role it can play in your cloud transformation strategy.

Read More Read More

GOTOpia 2021 – Platform Engineering as a (Community) Service

February 9, 2021 | Cloud Native, DevOps, Kubernetes

GOTOpia 2021 – Platform Engineering as a (Community) Service

Watch Nicki Watt’s talk on Platform Engineering as a (Community) Service at GOTOpia to learn what it takes to build a platform that is fit to serve the communities which will ultimately use it.

Read More Read More

WebAssembly – Where is it going?

December 11, 2020 | Cloud, Cloud Native, Kubernetes, Microservices

WebAssembly – Where is it going?

“WebAssembly is a safe, portable, low-level code format designed for efficient execution and compact representation.” – W3C

In this blog, I’ll cover the different applications of Wasm and WASI, some of the projects that are making headway, and the implications for modern architectures and distributed systems.

Read More Read More

It’s 2020, can I have my ML models now?

April 2, 2020 | Machine Learning

It’s 2020, can I have my ML models now?

Recent years have seen many companies consolidate all their data into a data lake/warehouse of some sort. Once it’s all consolidated, what next?

Many companies consolidate data with a field of dreams mindset – “build it and they will come”, however a comprehensive data strategy is needed if the ultimate goals of an organisation are to be realised: monetisation through Machine Learning and AI is an oft-cited goal. Unfortunately, before one rushes into the enticing world of machine learning, one should lay more mundane foundations. Indeed, in data science, estimates vary between 50% to 80% of the time taken is devoted to so-called data-wrangling. Further, Google estimates ML projects produce 5% ML code and 95% “glue code”. If this is the reality we face, what foundations are required before one can dive headlong into ML?

Read More Read More

Security, Usability & Cloud Data Services in Finance

March 20, 2020

Security, Usability & Cloud Data Services in Finance

Traditionally, Usability and Security have been set in opposition to each other: with tight security, we end up with painful user experience. In this blog, Guy focuses on financial services as an exemplar of how we can introduce usability into a vertical with challenging security and compliance requirements.

Read More Read More

Evolutionary challenges faced by VC funded organisations

November 13, 2019 | Software Consultancy

Evolutionary challenges faced by VC funded organisations

Pioneering and pushing technology boundaries – pretty much a given nowadays for the software-driven startup. Here are some insights we’ve observed working with a number of venture capital (VC) companies who have managed to navigate the choppy waters and successfully grow their business including winning further investment along the way.

With our deep hands-on technical expertise and pragmatic focus, OpenCredo has become a natural software acceleration partner for VC funded organisations who are looking to deliver tangible value as effectively as possible. We’ve been brought in to work alongside these innovators at various stages of their journey. As such we’ve gained an appreciation for and acquired, first-hand insight into some of the pressures and challenges faced. From getting and securing that next round of funding, to grappling with the technical decisions and challenges inherent in sensibly evolving offerings to accommodate future growth and scaling.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Why Upgrading to Terraform 0.12+ Should be a Priority

October 3, 2019 | Cloud, DevOps, Hashicorp, Open Source

Why Upgrading to Terraform 0.12+ Should be a Priority

Terraform 0.12 in recent years has emerged as the de facto standard with regards to defining and managing cloud infrastructure. It is one of four primary tools offered by HashiCorp, (Terraform, Vault, Consul and Nomad) and underpins the workflows that make up their Cloud Operating Model.

Since its first release in 2014, the wider Terraform community has embraced frequent releases and this past year has been no exception. HashiCorp announced the release of Terraform 0.12 in May 2019 and as of writing this post the official release is 0.12.9.

Read More Read More

Evolving Your Architecture Whilst Still Keeping The Lights On

September 12, 2019 | Cloud Native, Microservices, Software Consultancy

Evolving Your Architecture Whilst Still Keeping The Lights On

As a technology leader, you’ll be aware that competitive pressures and shifting business requirements are driving changes in the technical architectures of many organisations. This means you need a new strategic approach based on the ability to continually evolve elements of your systems and architectures.

Read More Read More

Securing Kafka using Vault PKI

February 20, 2019 | DevOps, Hashicorp, Kafka, Open Source

Securing Kafka using Vault PKI

Creating and managing a Public Key Infrastructure (PKI) could be a very straightforward task if you use appropriate tools. In this blog post, I’ll cover the steps to easily set up a PKI with Vault from HashiCorp, and use it to secure a Kafka Cluster.

Read More Read More

HA Prometheus – The Thanos Evolution

February 5, 2019 | Cloud Native, DevOps, Kubernetes, Microservices, Open Source

HA Prometheus – The Thanos Evolution

While Prometheus has fast become the standard for monitoring in the cloud, making Prometheus highly available can be tricky. This blog post will walk you through how to do this using the open source tool Thanos.

Read More Read More

10 Pieces of Borrowed Wisdom for Effective Tech Consultancy

August 29, 2018 | Software Consultancy

10 Pieces of Borrowed Wisdom for Effective Tech Consultancy

My last year or so here at OpenCredo has involved a very well-supported first foray into tech consultancy. Different engagements pose both unique, as well as familiar challenges for me as a consultant, all of which played a part in shaping and moulding the way I understand and approach problems. This blog is a brief collation of wisdom that’s helped me the most during this adventure; gained by learning the hard way, as well as that acquired from mentoring and colleagues who have gone before. The shared wisdom has made me a much more effective consultant, and kept me sane in the process, for which I’m very thankful.

Read More Read More

Self-testing infrastructure-as-code

May 31, 2018 | DevOps

Self-testing infrastructure-as-code

As traditional operations has embraced the concept of code, it has benefited from ideas already prevalent in developer circles such as version control. Version control brings the benefit that not only can you see what the infrastructure was, but you can also get reviews of changes by your peers before the change is made live; known to most developers as Pull Request (PR) reviews.

Read More Read More

Google Cloud Functions with Terraform

April 5, 2018 | Cloud, DevOps, Hashicorp, Terraform Provider

Google Cloud Functions with Terraform

Google Cloud Functions is the Google Cloud Platform (GCP) function-as-a-service offering. It allows you to execute your code in response to event triggers – HTTP, PubSub and Storage. While it currently only supports Node.js code for execution, it has proved very useful for running low-frequency operational tasks and other batch jobs in GCP.

Read More Read More

Fargate As An Enabler For Serverless Continuous Delivery

February 14, 2018 | Cloud

Fargate As An Enabler For Serverless Continuous Delivery

AWS Announced a few new products for use with containers at RE:Invent 2017 and of particular interest to me was a new Elastic Container Service(ECS) Launch type, called Fargate

Prior to Fargate, when it came to creating a continuous delivery pipeline in AWS, the use of containers through ECS in its standard form, was the closest you could get to an always up, hands off, managed style of setup. Traditionally ECS has allowed you to create a configured pool of “worker” instances, with it then acting as a scheduler, provisioning containers on those instances.

 

Read More Read More

A Pragmatic Introduction to Machine Learning for DevOps Engineers

January 23, 2018 | Data Engineering, DevOps

A Pragmatic Introduction to Machine Learning for DevOps Engineers

Machine Learning is a hot topic these days, as can be seen from search trends. It was the success of Deepmind and AlphaGo in 2016 that really brought machine learning to the attention of the wider community and the world at large.

Read More Read More

Writing a custom JupyterHub Spawner

January 11, 2018 | Data Engineering

Writing a custom JupyterHub Spawner

The last few years have seen Python emerge as a lingua franca for data scientists. Alongside Python we have also witnessed the rise of Jupyter Notebooks, which are now considered a de facto data science productivity tool, especially in the Python community. Jupyter Notebooks started as a university side-project known as iPython in circa 2001 at UC Berkeley.

Read More Read More

DevOpsDays Riga 2017 (Video): Technical and Project Management
Detecting stolen AWS credential usage with Apache Spark – Webinar Recording

May 22, 2017 | Data Analysis

Detecting stolen AWS credential usage with Apache Spark – Webinar Recording

As a final piece of our recent blog series about Apache Spark on 16 May we have presented details of a use-case about using Spark Structured Streaming to generate real-time alerts of suspicious activity in an AWS-based infrastructure.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Testing a Spark Application

May 9, 2017 | Cassandra

Testing a Spark Application

Data analytics isn’t a field commonly associated with testing, but there’s no reason we can’t treat it like any other application. Data analytics services are often deployed in production, and production services should be properly tested. This post covers some basic approaches for the testing of Cassandra/Spark code. There will be some code examples, but the focus is on how to structure your code to ensure it is testable!

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Deploy Spark with an Apache Cassandra cluster

May 2, 2017 | Cassandra, Data Engineering

Deploy Spark with an Apache Cassandra cluster

My recent blogpost I explored a few cases where using Cassandra and Spark together can be useful. My focus was on the functional behaviour of such a stack and what you need to do as a developer to interact with it. However, it did not describe any details about the infrastructure setup that is capable of running such Spark code or any deployment considerations. In this post, I will explore this in more detail and show some practical advice in how to deploy Spark and Apache Cassandra.

Read More Read More

New Blog Series: Spark – The Pragmatic Bits

April 25, 2017 | Cassandra, Data Analysis, Data Engineering

New Blog Series: Spark – The Pragmatic Bits

Apache Spark is a powerful open source processing engine which is fast becoming our technology of choice for data analytic projects here at OpenCredo. For many years now we have been helping our clients to practically implement and take advantage of various big data technologies including the like of Apache Cassandra amongst others.

 

 

Read More Read More

Data Analytics using Cassandra and Spark

March 23, 2017 | Cassandra, Data Analysis, Data Engineering

Data Analytics using Cassandra and Spark

In recent years, Cassandra has become one of the most widely used NoSQL databases: many of our clients use Cassandra for a variety of different purposes. This is no accident as it is a great datastore with nice scalability and performance characteristics.

However, adopting Cassandra as a single, one size fits all database has several downsides. The partitioned/distributed data storage model makes it difficult (and often very inefficient) to do certain types of queries or data analytics that are much more straightforward in a relational database.

Read More Read More

Google Cloud Spanner: our first impressions

March 7, 2017 | Data Analysis, GCP

Google Cloud Spanner: our first impressions

Google has recently made its internal Spanner database available to the wider public, as a hosted solution on Google Cloud. This is a distributed relational/transactional database used inside for various Google projects (including F1, the advertising backend), promising high throughput, low latency and 99.999% availability. As such it is an interesting alternative to many open source or other hosted solutions. This whitepaper gives a good theoretical introduction into Spanner.

Read More Read More

Everything you need to know about Cassandra Materialized Views

February 16, 2017 | Cassandra

Everything you need to know about Cassandra Materialized Views

One of the default Cassandra strategies to deal with more sophisticated queries is to create CQL tables that contain the data in a structure that matches the query itself (denormalization). Cassandra 3.0 introduces a new CQL feature, Materialized Views which captures this concept as a first-class construct.

Read More Read More

The Three ‘R’s of Distributed Event Processing

January 25, 2017 | Cassandra

The Three ‘R’s of Distributed Event Processing

One of the simplest and best-understood models of computation is the Finite State Machine (FSM). An FSM has fixed range of states it can be in, and is always in one of these states. When an input arrives, this triggers a transition in the FSM from its current state to the next state. There may be several possible transitions to several different states, and which transition is chosen depends on the input.

Read More Read More

What I Don’t Like About Error Handling in Go, and How to Work Around It

January 23, 2017 | Data Analysis

What I Don’t Like About Error Handling in Go, and How to Work Around It

More often than not, people who write Go have some sort of opinion on its error handling model. Depending on your experience with other languages, you may be used to different approaches. That’s why I’ve decided to write this article, as despite being relatively opinionated, I think drawing on my experiences can be useful in the debate. The main issues I wanted to cover are that it is difficult to force good error handling practice, that errors don’t have stack traces, and that error handling itself is too verbose.

Read More Read More

Cassandra – The Good, The Bad and the Ugly Webinar Recording

October 10, 2016 | Cassandra

Cassandra – The Good, The Bad and the Ugly Webinar Recording

In the culmination of our blog series on the topic, on October 6th 2016 OpenCredo Consultants Dominic Fox, Alla Babkina and Guy Richardson, and hosted by Marco Cullen, presented the common design and implementation issues that they have come across in real-world Apache Cassandra deployments.

Read More Read More

Fulfilling the promise of Apache Cassandra performance

August 24, 2016 | Cassandra

Fulfilling the promise of Apache Cassandra performance

At OpenCredo we are seeing an increase in adoption of Apache Cassandra as a leading NoSQL database for managing large data volumes, but we have also seen many clients experiencing difficulty converting their high expectations into operational Cassandra performance. Here we present a high-level technical overview of the major strengths and limitations of Cassandra that we have observed over the last few years while helping our clients resolve the real-world issues that they have experienced.

Read More Read More

Key Takeaways from the DevOps Enterprise Summit (#DOES16) EU Conference

July 3, 2016 | DevOps

Key Takeaways from the DevOps Enterprise Summit (#DOES16) EU Conference

Several of us from the OpenCredo team were in attendance at the inaugural EU edition of the DevOps Enterprise Summit conference. We have been big fans of the two previous US versions, and have watched the video recordings of talks (2014, 2015) with keen interest as many of our DevOps transformation clients are very much operating in the ‘enterprise’ space.

Read More Read More

You Are Ignoring Non-­functional Testing

June 15, 2016 | Software Consultancy

You Are Ignoring Non-­functional Testing

It’s as simple as that – and as a consultant, it’s a problem I see all the time. Testing is always focused on functional testing. Non-functional testing, by comparison, is treated like a second class citizen. This means that functional requirements get refined, and non-functional requirements are ignored until the very end.

Read More Read More

The Seven Deadly Sins of Project Managers

May 27, 2016 | Software Consultancy

The Seven Deadly Sins of Project Managers

People make mistakes. People can improve. People need to be told of their mistakes. Surprisingly project managers are people too… I know… in some cases it is really hard to believe it; and then they can improve too! Shocking! I have more than 15 years working in IT, occupying different roles from software development to project management in three different countries. I’ve seen really bad project managers and some really good project managers; Yes, they DO exist!

Read More Read More

Securing Terraform state with Vault

April 2, 2016 | Terraform Provider

Securing Terraform state with Vault

When it comes to automating the creation of infrastructure in cloud providers, Terraform (version at time of writing 0.6.14) has become one of my core go to tools in this space. It provides a fantastic declarative approach to describing the resources you want, and then takes care of making it so for you, keeping track of the state in either a local file or a remote store of some sort. Various bits of sensitive data is often provided as input to terraform.

Read More Read More

Kotlin: a new JVM language you should try

March 3, 2016 | Software Consultancy

Kotlin: a new JVM language you should try

JetBrains (the people behind IntelliJ IDEA) have recently announced  the first RC for version 1.0 of Kotlin, a new programming language for the JVM. I say ‘new’, but Kotlin has been in the making for a few years now, and has been used by JetBrains to develop several of their products, including Intellij IDEA. The company open-sourced Kotlin in 2011, and have worked with the community since then to make the language what it is today.

Read More Read More

Versioning a Microservice System with git

March 2, 2016 | Microservices

Versioning a Microservice System with git

Microservice-style software architectures have many benefits: loose coupling, independent scalability, localised failures, facilitating the usage of polyglot data persistence tools or multiple programming languages.

However, they also introduce other challenges. A major one is the fact that the end-user functionality of the system will ultimately emerge as a composition of multiple services. This significantly increases the complexity of deploying the system. In addition, because we lose the concept of “versions” of the system, it becomes harder to answer questions like “what capabilities are in production?” and “when is a new feature considered ‘done’?”.

Read More Read More

DevOps and IT Operations

January 29, 2016 | DevOps

DevOps and IT Operations

DevOps is 2016’s tech holy grail – unified development and operations, both working to deliver what the business needs, quickly, reliably, and adaptably. Done well, DevOps transforms the way organisations work; it helps break down barriers between tech teams, and between technology and the rest of the business. Good DevOps is the antidote to increasing segmentation and specialisation within companies. With the promised benefits, is it any wonder that senior managers are pushing for it in organisations spanning all sizes and industries?

Read More Read More

Akka Typed brings type safety to Akka framework

January 18, 2016 | Software Consultancy

Akka Typed brings type safety to Akka framework

Last time in this series I summarised all the Akka Persistence related improvements in Akka 2.4. Since then Akka 2.4.1 has been released with some additional bug fixes and improvements so perhaps now is a perfect time to pick up this mini-series and introduce some other new features included in Akka 2.4.x.

Read More Read More

Facebook’s React, and the Signal:Noise Ratio

January 15, 2016 | Software Consultancy

Facebook’s React, and the Signal:Noise Ratio

So, you’ve started to hear a lot about React, the Javascript library developed by Facebook, but is it something you need to investigate? It’s time to distil the signal from the noise, position React amongst its rivals, and provide an indication of where it currently would – and would not – be a suitable fit.

Read More Read More

The Seven Deadly Sins of Microservices (Redux)

January 8, 2016 | Microservices

The Seven Deadly Sins of Microservices (Redux)

Many of our clients are in the process of investigating or implementing ‘microservices’, and a popular question we often get asked is “what’s the most common mistake you see when moving towards a microservice architecture?”. We’ve seen plenty of good things with this architectural pattern, but we have also seen a few recurring issues and anti-patterns, which I’m keen to share here.

Read More Read More

The Risk of DevOps Tools: Automated Failure

December 16, 2015 | Cloud, DevOps

The Risk of DevOps Tools: Automated Failure

The Promise of DevOps Tools

In the rush to embrace DevOps, many organisations seek out tools to help them achieve DevOps nirvana; the magical tools that will unify Development and Operations, stop the infighting, and ensure collaboration. This search for tools to solve problems exists in many domains, but seems particularly prevalent in IT (it may be real, or a reflection of my exposure to IT). The temptation to embrace new tools as a panacea is high, because the problems in IT seem so pervasive and persistent.

Read More Read More

RAML 1.0 promotes reusability and standardisation

November 25, 2015 | Microservices

RAML 1.0 promotes reusability and standardisation

In May a 1.0 release of RAML (RESTful API Markup Language) has been announced delivering a few much welcome additions in the RAML 1.0 specification. This major release marks an important milestone in the evolution of RAML and indicates the team behind the specification is confident this release delivers the comprehensive set of tools for developing RESTful APIs. I’ve been using RAML 0.8 for several months now and have enjoyed the simplicity and productivity it offers for designing and documenting APIs. I must say I’m quite pleased with the changes introduced in the new release and would like to review those I consider particularly useful.

Read More Read More

Webinar invite: Deploying HashiCorp Vault for secure cloud deployments
Reusing ansible roles with private git repos and dependency management

November 4, 2015 | Software Consultancy

Reusing ansible roles with private git repos and dependency management

Writing reusable roles for Ansible is not an easy task but one that’s worth doing. This post should walk you through the basics of writing reusable roles with dependencies backed by public and private git repositories.

Read More Read More

Shadow IT

October 30, 2015 | Cloud, DevOps

Shadow IT

In some companies, the inevitable rapidly became accepted as the way to do things, and both development and IT operations worked together to figure out how to collaborate on building systems that satisfied development’s desire for change, and operations desire for stability. Outsourcing infrastructure, and all it implied, gave rise to Devops – the unification of business needs, developer delivery, and operational capacity – but it also gave rise to something else, in companies where the operations teams weren’t quite as quick to move – Shadow IT.

Read More Read More

What’s new in Akka Persistence 2.4

October 28, 2015 | Software Consultancy

What’s new in Akka Persistence 2.4

Let’s have a quick look at the most interesting changes and new features that are now available to Akka users. As there are many new features to highlight in the new Akka release I will focus on those related to Akka Persistence first and cover other areas in a separate post.

Read More Read More

DevOps is Transformative

October 12, 2015 | DevOps

DevOps is Transformative

The Pre-DevOps Environment

DevOps is transformative. This (hopefully) won’t be true forever, but it is for now. While the modern management practices of separating development and operations (and to a lesser extent, everyone else) prevail, the tearing down of the walls that separate them will remain transformative. In company after company, management and front-line staff are coming to realise that keeping functions separate, which are inherently interdependent, is a model for blame, shifted responsibility, and acrimony. It’s easy to divvy-up a company up based on function. To many people, it seems the most logical way to do it. Ops does operations, Dev does development, Marketing markets, etc. It seems much harder to do it any other way. So why do it?

Read More Read More

Change is coming to IT

September 21, 2015 | DevOps

Change is coming to IT

The drive for change comes from, everywhere. Within IT departments, developers are asking for more control, and want faster response times than can be provided by traditional gatekeeper IT services. Management hears about new technologies allowing more rapid deployments, better ROI, reduction in costs, increased efficiency, and increased scalability, and they want it for their customers. On top of that, systems age, technologies lose their lustre, and technical debt builds up, while it simultaneously gets harder to recruit people from a shrinking talent pool, all of which creates a case for change from within the technology itself.

Read More Read More

Terraform Infrastructure Design Patterns

September 14, 2015 | Cloud, DevOps

Terraform Infrastructure Design Patterns

If you are operating in the programmable infrastructure space, you will hopefully have come across Terraform, a tool from HashiCorp which is primarily used to manage infrastructure resources such as virtual machines, DNS names and firewall settings across a number of public and private providers (AWS, GCP, Azure, …).

Read More Read More

Designing a REST API with fine-grained resources, HATEOAS and HAL

August 12, 2015 | Microservices

Designing a REST API with fine-grained resources, HATEOAS and HAL

Over the last few months one of my main projects at OpenCredo has involved creating various microservices which are interacted with via REST. We’ve been working with a relatively rich domain model, which in turn has presented a lot of challenges in how to design our various resources. This blog post aims to summarise various techniques and practices which I’ve found helpful in overcoming these challenges.

Read More Read More

Boot my (secure)->(gov) cloud

August 10, 2015 | Cloud, Software Consultancy

Boot my (secure)->(gov) cloud

As a company, we at OpenCredo are heavily involved in automation and devOps based work, with a keen focus on making this a seamless experience, especially in cloud based environments. We are currently working within HMRC, a UK government department to help make this a reality as part of a broader cloud broker ecosystem project. In this blog post, I look to provide some initial insight into some of the tools and techniques employed to achieve this for one particular use case namely:
With pretty much zero human intervention, bar initiating a process and providing some inputs, a development team from any location, should be able to run “something”, which, in the end, results in an isolated, secure set of fully configured VM’s being provisioned within a cloud provider (or providers) of choice.

Read More Read More

Embracing Disruptive Innovation: OpenCredo Partners with Google

August 5, 2015 | Cloud, GCP, Kubernetes

Embracing Disruptive Innovation: OpenCredo Partners with Google

Why OpenCredo partnered with Google

Recently OpenCredo chose to partner with Google in order to share knowledge and resources around the Google Cloud Platform offerings. Our clients come in many shapes and sizes, but typically all of them realise three disruptive truths of the modern IT industry: the (economic) value of cloud; the competitive advantage of continuous delivery; and the potential of hypothesis and data-driven product development to increase innovation (as popularised by the Lean Startup / Lean Enterprise motto of ‘build, measure, learn’).

Read More Read More

A deep dive into Angular 2.0

July 8, 2015 | Software Consultancy

A deep dive into Angular 2.0

I was quite excited around autumn last year when Google started to work on a new version of Angular (Angular 2.0) which promised to revolutionise web development. There were rumours that Angular 2.0 wouldn’t be backward compatible with its predecessor, and would be written in Google’s AtScript which is a JavaScript based language on top of Microsoft’s TypeScript. The lack of backwards compatibility raised some concerns, especially for the clients that we had used Angular at. But, lets not get ahead of ourselves here….

Read More Read More

Asynchronous Cloud bootstrapping with Terraform, Cloud-Init & Puppet

June 23, 2015 | Cloud, DevOps, Terraform Provider

Asynchronous Cloud bootstrapping with Terraform, Cloud-Init & Puppet

Working with OpenCredo clients, I’ve noticed that even if you are one of the few organisations that can boast ‘Infrastructure as Code’, perhaps it’s only true of your VMs, and likely you have ‘bootstrap problems’. What I mean by this, is that you require some cloud-infrastructure to already be in place before your VM automation can go to work.

Read More Read More

Watch ‘Elastic Analytics with Spark, Mesos and Docker’

May 13, 2015 | Software Consultancy

Watch ‘Elastic Analytics with Spark, Mesos and Docker’

Listen to Brenden Matthews discuss Elastic Analytics with Spark, Mesos and Docker as filmed at the most recent London Mesos User Group Meetup.

In this talk, Brenden Matthews discusses how he provided elastic analytics to Airbnb and how the Mesosphere DCOS can easily bring the same type of infrastructure to your own environments.

Read More Read More

Join the London Mesos User Group to find out more about Elastic Analytics
Supporting the G-Cloud 6 framework …
How to Write Your Own Apache Mesos Framework?

February 16, 2015 | Software Consultancy

How to Write Your Own Apache Mesos Framework?

Apache Mesos is often explained as being a kernel for the data-centre; meaning that cluster resources (CPU, RAM, …) are tracked and offered to “user space” programs (i.e. frameworks) to do computations on the cluster.

Read More Read More

Traits with Java 8 Default Methods

January 30, 2015 | Software Consultancy

Traits with Java 8 Default Methods

When I first started programming in Scala a few years ago, Traits was the language feature I was most excited about. Indeed, Traits give you the ability to compose and share behaviour in a clean and reusable way. In Java, we tend deal with the same concerns by grouping crosscutting behaviour in abstract base classes that are subsequently extended every time we need to access shared behaviour in part or in total.

Read More Read More

November 19, 2014 | Microservices

The 7 Deadly Sins of Microservices

Undeniably, there is a growing interest in microservices as we see more organisations, big and small, evaluating and implementing this emerging approach. Despite its apparent novelty, most concepts and principles underpinning microservices are not exactly new – they are simply proven and commonsense software development practices that now need to be applied holistically and at a wider scale, rather than at the scale of a single program or machine. These principles include separation of concerns, loose coupling, scalability and fault-tolerance.

Read More Read More

Configuration Management with Flexible Contexts

August 2, 2013 | Software Consultancy

Configuration Management with Flexible Contexts

Configuration management was born in the pre-cloud era. Remember the days when acquiring a super powerful multi core server felt like winning the jackpot? Infrastructure was a slightly different place back then. Yet for all the recent developments in DevOps, its legacy is still with us.

Read More Read More

Model Matters: Graphs, Neo4j and the Future

February 25, 2013 | Neo4j

Model Matters: Graphs, Neo4j and the Future

As part of our work, we often help our customers choose the right datastore for a project. There are usually a number of considerations involved in that process, such as performance, scalability, the expected size of the data set, and the suitability of the data model to the problem at hand.

This blog post is about my experience with graph database technologies, specifically Neo4j. I would like to share some thoughts on when Neo4j is a good fit but also what challenges Neo4j faces now and in the near future.

Read More Read More

Introducing CloudCredo!
API Testing with Cucumber

February 4, 2013 | Software Consultancy

API Testing with Cucumber

This API will in future be used by a mobile client and by third parties, making it important to verify that it is functionally correct as well as clearly documented.

An additional requirement in our case is for the tests to form a specification for the API to allow front and back end developers agree on the format in advance. This is something that BDD excels at, making it natural to continue to use Cucumber. This post will focus on the difficulties of attaining the appropriate level of abstraction with Cucumber while retaining the technical detail required for specification.

Read More Read More

Withstanding the test of time

December 18, 2012 | Software Consultancy

Withstanding the test of time

The first thing most people think of when they start a project with the good intentions of test driven development is: write a test first. That’s great, and something I would fully encourage. However, diving in to writing tests without forethought, especially on large projects with a lot of developers can lead to new problems that TDD is not going to solve. With some upfront thinking (but not big upfront design!) a large team can avoid problems later down the line by considering some important and desirable traits of a large and rapidly changing test suite.

Read More Read More

48 hours – The Pursuit of Proper Performance

February 13, 2012 | Software Consultancy

48 hours – The Pursuit of Proper Performance

Recently we were approached by a client to do some performance testing of the web application they had written. The budget allowed two days for this task. Ok. No problem. Yes, we can. Naturally I had one or another question though…

Read More Read More

Practical behaviour driven development – Maintainable Acceptance Tests

February 3, 2012 | Software Consultancy

Practical behaviour driven development – Maintainable Acceptance Tests

Behaviour driven development (BBD)

Creating maintainable acceptance test

In this post we will look at the creation of a suite of maintainable acceptance tests, identifying some key issues as encountered on real projects and suggesting solutions.This post assumes some knowledge of behaviour driven development (BDD). If you are new to this concept, I’d recommend Dan North’s blog as a starting point, particularly his post introducing the concept of BDD.The idea of human readable acceptance testing championed by BDD enthusiasts is one that continues to grow in popularity, and it’s easy to see why.

Read More Read More

Neo4j: Super-Nodes and Indexed Relationships, Part I

June 3, 2011 | Neo4j

Neo4j: Super-Nodes and Indexed Relationships, Part I

Neo4J is one of the first graph databases to appear on the global market. Being open source, in addition to its power and simplicity in supporting graph data model it represents good choice for production-ready graph database.

However, there has been one area I have struggled to get good-enough performance from Neo4j recently – super nodes.

Read More Read More