Open Credo

80 items found: Search results for "update" in all categories x

New features in Cassandra 2.0 – Lightweight Transactions on Update

February 17, 2014 | Cassandra

New features in Cassandra 2.0 – Lightweight Transactions on Update

In our previous posts we gave an overview of Cassandra’s new compare-and-set (lightweight transaction) commands and a more detailed look into the API for using them when inserting new rows into the database.

In this third post, we are going to cover update statements. We recommend reading the previous posts, as there are some details which are the same for inserts and updates which are not repeated here.

Read More Read More

The 2023 Mayor’s Business Climate Challenge (BCC) – Final Part

March 5, 2024 | Blog, Culture, News

The 2023 Mayor’s Business Climate Challenge (BCC) – Final Part

Learn more about our efforts and our progress towards becoming an environmentally friendly company for the Mayor’s Business Climate Challenge (BCC) in 2023 in this final update.

Read More Read More

Use Case: A Kafka Backup Solution

February 22, 2024 | Blog, Kafka

Use Case: A Kafka Backup Solution

Check out Peter Vegh’s latest blog where he explores a bespoke Kafka backup framework, discussing the approach, architecture and design process to implement and deliver the solution.

Read More Read More

Event Driven Load Testing

October 12, 2023 | Blog, Platform Engineering

Event Driven Load Testing

Check out our latest blog “Event Driven Load Testing” which explores how, through some smart automation techniques, testing strategies can be adapted to support scale-up organisations where there are potentially many disparate teams needing to work together.

Read More Read More

The Why’s, What’s and How’s of Kubernetes Operators

September 27, 2023 | Blog, Kubernetes

The Why’s, What’s and How’s of Kubernetes Operators

Learn to create your first Kubernetes operator by checking out our Senior Consultant Michal Tusnio’s latest blog, “Kubernetes Operators – Whys, Hows and Whats” where he takes you on a journey from zero to operator.

Read More Read More

The 2023 Mayor’s Business Climate Challenge (BCC) – Part 2

September 19, 2023 | Blog, Culture, News

The 2023 Mayor’s Business Climate Challenge (BCC) – Part 2

In this blog we share the latest developments in our efforts to create a more sustainable business as part of The 2023 Mayor’s Business Climate Challenge.

Read More Read More

Why should you upgrade your Terraform provider to the latest version of the API

August 17, 2023 | Blog, Terraform Provider

Why should you upgrade your Terraform provider to the latest version of the API

Check out John Sharpe and Will May’s latest blog where they give suggestions for Terraform Provider authors who are thinking about upgrading from SDKv2 to Framework

Read More Read More

TL;DR – Computational Governance in Data Mesh with OPA

April 21, 2023 | Blog, Data Engineering, Data Governance, Data Mesh, OPA

TL;DR – Computational Governance in Data Mesh with OPA

Check out Mateus Pimenta’s TL;DR video to learn how federated computational governance could be implemented using Open Policy Agent (OPA) and policy-as-code to support a successful Data Mesh architecture.

Read More Read More

Ingesting Big Data into Neo4j – Part 3

March 9, 2023 | Blog, Data Analysis, Neo4j

Ingesting Big Data into Neo4j – Part 3

Check out the last part of Ebru Cucen and Fahran Wallace’s blog series, in which they discuss their experience ingesting 400 million nodes and a billion relationships into Neo4j and what they discovered along the way.

Read More Read More

Ingesting Big Data into Neo4j – Part 2

February 16, 2023 | Blog, Data Analysis, Neo4j

Ingesting Big Data into Neo4j – Part 2

Check out Part 2 of Ebru Cucen and Fahran Wallace’s blog series, in which they discuss their experience ingesting 400 million nodes and a billion relationships into Neo4j and what they discovered along the way.

Read More Read More

Ingesting Big Data into Neo4j – Part 1

January 26, 2023 | Blog, Data Analysis, Neo4j

Ingesting Big Data into Neo4j – Part 1

Fahran Wallace and Ebru Cucen’s most recent blog post is part 1 of a three-part series. They investigate how OpenCredo ingested 400 million nodes with a billion relationships into Neo4j.

Read More Read More

Controlling Kafka Data Flows using Open Policy Agent

August 2, 2022 | Blog, Kafka

Controlling Kafka Data Flows using Open Policy Agent

Read Matt Farrow’s blog as he explores the potential for using Open Policy Agent to filter and mask data being sent to and read from Apache Kafka.

Read More Read More

HackCredo – An Internal OpenCredo Hackathon

July 5, 2022 | Blog, Culture

HackCredo – An Internal OpenCredo Hackathon

As we are passionate about using technology to solve problems, we are thrilled to share with you our internal competition, “HackCredo.” Read on to learn more about the competition, the groups, and the winners.

Read More Read More

Making Sense of Data with RDF* vs. LPG

January 31, 2022 | Blog, Data Engineering

Making Sense of Data with RDF* vs. LPG

There are two camps of Graph database, one side is RDF, where they are strict with their format, and somewhat limited for their extensibility. The other side is LPG, where they can define labels to the relationships. With its recent extension, RDF now allows users to add properties, thus becoming RDF*. In this blog, Ebru explores the structural and performance differences between LPG and RDF*.

Read More Read More

What you might have missed in Kubernetes 1.22 release

December 5, 2021 | Cloud, Kubernetes

What you might have missed in Kubernetes 1.22 release

Kubernetes’ second release in 2021, version 1.22, has been out for a little while now and with 1.23 on its way, we thought we’d take a look back. Kubernetes 1.22 was a highly comprehensive release with 53 enhancements in all three graduation levels: 13 features have graduated to stable, 24 enhancements reached beta status, and 16 new features have been accepted into the alpha stage. 

The latest version has some noteworthy security features such as running Kubelet without root access, pod security policies, and seccomp. There are also a couple of deprecated and removed APIs. In this blog, we’ll discuss the significant changes in v1.22, as well as how to handle the removed APIs.

Read More Read More

Exploring How Policy-as-Code and OPA Fit into the K8s World

November 4, 2021 | Kubernetes

Exploring How Policy-as-Code and OPA Fit into the K8s World

We always read that ‘security is everyone’s responsibility’. For any organisation, big or small, security should always be the primary concern—not a mere afterthought. In terms of Kubernetes, securing a cluster is challenging because it has so many moving parts and, apart from securing our Kubernetes environment, we also want to control what an end-user can do in our cluster.

To achieve these goals, we can start with the built-in features provided by Kubernetes like Role-Based Access Control (RBAC), Network Policies, Secrets Management, and Pod Security Policies (PSP). But we know these features are not enough. For example, we may want specific policies like ‘all pods must have specific labels’. And even if we have the policies in place, the next big question is how to enforce them on our Kubernetes cluster in an easy and repeatable manner.

In this blog post, we’ll address this challenge and other questions pertaining to OPA and how it can integrate into Kubernetes.

Read More Read More

Running the Cloud from your Kubernetes Cluster

September 2, 2021 | Blog, Cloud, Kubernetes

Running the Cloud from your Kubernetes Cluster

In this blog, Stuart compares the new approach of deploying cloud resources as Kubernetes custom resources rather than the (now) typical approach using Terraform – or cloud specific: CloudFormation (AWS), Deployment Manager (GCP). He also identifies what resources are suitable for this approach and which ones are not.

Read More Read More

Machine Learning at scale: first impressions of Kubeflow

April 20, 2021 | Data Engineering, Machine Learning, Software Consultancy

Machine Learning at scale: first impressions of Kubeflow

Our recent client was a Fintech who had ambitions to build a Machine Learning platform for real-time decision making. The client had significant Kubernetes proficiency, ran on the cloud, and had a strong preference for using free, open-source software over cloud-native offerings that come with lock-in. Several components were spiked with success (feature preparation with Apache Beam and Seldon for model serving performed particularly strongly). Kubeflow was one of the next technologies on our list of spikes, showing significant promise at the research stage and seemingly a good match for our client’s priorities and skills.

That platform slipped down the client’s priority list before completing the research for Kubeflow, so I wanted to see how that project might have turned out. Would Kubeflow have made the cut?

 

Read More Read More

WebAssembly – Where is it going?

December 11, 2020 | Cloud, Cloud Native, Kubernetes, Microservices

WebAssembly – Where is it going?

“WebAssembly is a safe, portable, low-level code format designed for efficient execution and compact representation.” – W3C

In this blog, I’ll cover the different applications of Wasm and WASI, some of the projects that are making headway, and the implications for modern architectures and distributed systems.

Read More Read More

Automation of complex IT systems

May 14, 2020 | Blog, DevOps

Automation of complex IT systems

At the time of this post, the UK is making steps to exit from an unprecedented lockdown measures for the Coronavirus. Much of the UK workforce are still making efforts to work-from-home with mainly key workers operating – at risk – in public. Many industries have shut down completely. Consequently, many businesses are reflecting on what happens next and how do we better mitigate future pandemic events?

Read More Read More

3 Highlights from CloudNative London 2019 (Day 1)

October 1, 2019 | Cloud, Cloud Native, Culture

3 Highlights from CloudNative London 2019 (Day 1)

One of the benefits we have working at OpenCredo (OC) is the opportunity to both attend and speak (although not on this occasion) at conferences. For some of you, this may be pretty common, but OC was actually the first to offer me this as part of a broader learning and development plan.

Cloud-native development and delivery is a core area of expertise for OC and we are always looking for what’s new and interesting in this space. So when I was offered the chance to go to CloudNative London it seemed like a good place to start. With its diversity in topics and technologies, the conference provided a perfect opportunity to collaborate and hear from others in the industry and what they are doing in this space.

Read More Read More

Evolving Your Architecture Whilst Still Keeping The Lights On

September 12, 2019 | Cloud Native, Microservices, Software Consultancy

Evolving Your Architecture Whilst Still Keeping The Lights On

As a technology leader, you’ll be aware that competitive pressures and shifting business requirements are driving changes in the technical architectures of many organisations. This means you need a new strategic approach based on the ability to continually evolve elements of your systems and architectures.

Read More Read More

Securing Kafka using Vault PKI

February 20, 2019 | DevOps, Hashicorp, Kafka, Open Source

Securing Kafka using Vault PKI

Creating and managing a Public Key Infrastructure (PKI) could be a very straightforward task if you use appropriate tools. In this blog post, I’ll cover the steps to easily set up a PKI with Vault from HashiCorp, and use it to secure a Kafka Cluster.

Read More Read More

Data Science on Steroids: Productionised Machine Learning as a Value Driver for Business

July 31, 2018 | Machine Learning

Data Science on Steroids: Productionised Machine Learning as a Value Driver for Business

Machine Learning, alongside a mature Data Science, will help to bring IT and business closer together. By leveraging data for actionable insights, IT will increasingly drive business value. Agile and DevOps practices enable the continuous delivery of business value through productionised machine learning models and software delivery.

Read More Read More

Self-testing infrastructure-as-code

May 31, 2018 | DevOps

Self-testing infrastructure-as-code

As traditional operations has embraced the concept of code, it has benefited from ideas already prevalent in developer circles such as version control. Version control brings the benefit that not only can you see what the infrastructure was, but you can also get reviews of changes by your peers before the change is made live; known to most developers as Pull Request (PR) reviews.

Read More Read More

Microservices Anti-patterns: It’s All About the People

April 18, 2018 | Microservices

Microservices Anti-patterns: It’s All About the People

Quite a few of the anti-patterns we observe today on microservices projects are strongly related to how people approach the problem. Given their nature, these anti-patterns tend to be deeply ingrained and self-sustaining. Addressing them starts with increased awareness and by changing ways of approaching the problem, rather than by the introduction of yet another technical tool or framework.

Read More Read More

Events and Commands: Two Faces of the Same Coin?

March 14, 2018 | Data Engineering, Microservices

Events and Commands: Two Faces of the Same Coin?

Events are obviously the fundamental building block of event-sourced systems. Commands are equally a common concept in such systems although the distinction between events and commands, if any, is not always clear. There are certainly varying views on what role each one should play.

Read More Read More

Fargate As An Enabler For Serverless Continuous Delivery

February 14, 2018 | Cloud

Fargate As An Enabler For Serverless Continuous Delivery

AWS Announced a few new products for use with containers at RE:Invent 2017 and of particular interest to me was a new Elastic Container Service(ECS) Launch type, called Fargate

Prior to Fargate, when it came to creating a continuous delivery pipeline in AWS, the use of containers through ECS in its standard form, was the closest you could get to an always up, hands off, managed style of setup. Traditionally ECS has allowed you to create a configured pool of “worker” instances, with it then acting as a scheduler, provisioning containers on those instances.

 

Read More Read More

Get your -aas in gear: Privatelink and PaaS democratisation on AWS

February 6, 2018 | Cloud

Get your -aas in gear: Privatelink and PaaS democratisation on AWS

Among the many announcements made at Re:Invent 2017 was the release of AWS Privatelink for Customer and Partner services. We believe that the opportunity signalled by this modest announcement may have an impact far broader than first impressions suggest.

Read More Read More

Riak, the Dynamo paper and life beyond Basho

August 8, 2017 | Cassandra

Riak, the Dynamo paper and life beyond Basho

Recently, the sad news has emerged that Basho, which developed the Riak distributed database, has gone into receivership. This would appear to present a problem for those who have adopted the commercial version of the Riak database (Riak KV) supported by Basho.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Just Cloud It: leading digital business innovation companies team up in cloud consulting joint venture
Data Analytics using Cassandra and Spark

March 23, 2017 | Cassandra, Data Analysis, Data Engineering

Data Analytics using Cassandra and Spark

In recent years, Cassandra has become one of the most widely used NoSQL databases: many of our clients use Cassandra for a variety of different purposes. This is no accident as it is a great datastore with nice scalability and performance characteristics.

However, adopting Cassandra as a single, one size fits all database has several downsides. The partitioned/distributed data storage model makes it difficult (and often very inefficient) to do certain types of queries or data analytics that are much more straightforward in a relational database.

Read More Read More

Google Cloud Spanner: our first impressions

March 7, 2017 | Data Analysis, GCP

Google Cloud Spanner: our first impressions

Google has recently made its internal Spanner database available to the wider public, as a hosted solution on Google Cloud. This is a distributed relational/transactional database used inside for various Google projects (including F1, the advertising backend), promising high throughput, low latency and 99.999% availability. As such it is an interesting alternative to many open source or other hosted solutions. This whitepaper gives a good theoretical introduction into Spanner.

Read More Read More

Some Uses For Type Aliases in Kotlin 1.1

February 28, 2017 | Software Consultancy

Some Uses For Type Aliases in Kotlin 1.1

As Kotlin’s 1.1 release draws closer, I’ve been looking at some of the new language features it supports. Type aliases may seem like a relatively minor feature next to coroutines, but as I will show in this blog post, they can open up a new programming idiom, particularly when combined with extension functions.

Read More Read More

Everything you need to know about Cassandra Materialized Views

February 16, 2017 | Cassandra

Everything you need to know about Cassandra Materialized Views

One of the default Cassandra strategies to deal with more sophisticated queries is to create CQL tables that contain the data in a structure that matches the query itself (denormalization). Cassandra 3.0 introduces a new CQL feature, Materialized Views which captures this concept as a first-class construct.

Read More Read More

The Three ‘R’s of Distributed Event Processing

January 25, 2017 | Cassandra

The Three ‘R’s of Distributed Event Processing

One of the simplest and best-understood models of computation is the Finite State Machine (FSM). An FSM has fixed range of states it can be in, and is always in one of these states. When an input arrives, this triggers a transition in the FSM from its current state to the next state. There may be several possible transitions to several different states, and which transition is chosen depends on the input.

Read More Read More

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

January 24, 2017 | Cloud

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

This blog aims to provide an end to end example of how you can automatically request, generate and install a free HTTPS/TLS/SSL certificate from Let’s Encrypt using Terraform. Let’s Encrypt is a free, automated, and open certificate authority (CA) aiming to make it super easy (and free – did I say free!) for people to obtain HTTPS (SSL/TLS) certificates for their websites and infrastructure. Under the hood, Let’s Encrypt implements and leverages an emerging protocol called ACME to make all this magic happen, and it is this ACME protocol that powers the Terraform provider we will be using. For more information on how Let’s Encrypt and the ACME protocol actually work, please see how Let’s Encrypt works.

Read More Read More

Common Problems with Cassandra Tombstones

September 27, 2016 | Cassandra, Data Engineering

Common Problems with Cassandra Tombstones

If there is one thing to understand about Cassandra, it is the fact that it is optimised for writes. In Cassandra everything is a write including logical deletion of data which results in tombstones – special deletion records. We have noticed that lack of understanding of tombstones is often the root cause of production issues our clients experience with Cassandra. We have decided to share a compilation of the most common problems with Cassandra tombstones and some practical advice on solving them.

Read More Read More

Simulating GitHub OAuth2 Login for Tests with JUnit, Webdriver & Hoverfly: Record, Sanitise, Playback and Assert

September 21, 2016 | DevOps

Simulating GitHub OAuth2 Login for Tests with JUnit, Webdriver & Hoverfly: Record, Sanitise, Playback and Assert

Sometimes, it can be difficult to write automated tests for parts of your application due to complexities introduced by an external dependency. It may be flaky or have some sort of rate limiting, or require sensitive information which we don’t want to expose outside of our production environment. To get around this, teams might take the approach of manually stubbing the service or using mocks – but the former is tedious and error prone, whereas the latter doesn’t test collaboration at all.

Read More Read More

How Not To Use Cassandra Like An RDBMS (and what will happen if you do)

September 15, 2016 | Cassandra

How Not To Use Cassandra Like An RDBMS (and what will happen if you do)

Cassandra isn’t a relational database management system, but it has some features that make it look a bit like one. Chief among these is CQL, a query language with an SQL-like syntax. CQL isn’t a bad thing in itself – in fact it’s very convenient – but it can be misleading since it gives developers the illusion that they are working with a familiar data model, when things are really very different under the hood.

Read More Read More

Patterns of Successful Cassandra Data Modelling

September 6, 2016 | Cassandra

Patterns of Successful Cassandra Data Modelling

A growing number of clients are asking OpenCredo for help with using Apache Cassandra and solving specific problems they encounter. Clients have different use cases, requirements, implementation and teams but experience similar issues. We have noticed that Cassandra data modelling problems are the most consistent cause of Cassandra failing to meet their expectations. Data modelling is one of the most complex areas of using Cassandra and has many considerations.

Read More Read More

New Blog Series: Cassandra – What You May Learn the Hard Way

August 26, 2016 | Cassandra

New Blog Series: Cassandra – What You May Learn the Hard Way

At OpenCredo we have been working with Cassandra since 2012 and we are big fans of both open source Apache Cassandra and the capabilities of DataStax Enterprise. Over the years we have collected a great deal of experience throughout the company on how to deliver the benefits of Cassandra in real world projects and have also seen some common pitfalls that businesses have fallen into.

Read More Read More

Kubernetes from scratch to AWS with Terraform and Ansible (part 3)

August 26, 2016 | Kubernetes

Kubernetes from scratch to AWS with Terraform and Ansible (part 3)

This post is the last of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible. To understand the goal of the project, you’d better start from the first part.

Read More Read More

Kubernetes from scratch to AWS with Terraform and Ansible (part 1)

August 26, 2016 | Kubernetes

Kubernetes from scratch to AWS with Terraform and Ansible (part 1)

This post is the first of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible.

Read More Read More

Fulfilling the promise of Apache Cassandra performance

August 24, 2016 | Cassandra

Fulfilling the promise of Apache Cassandra performance

At OpenCredo we are seeing an increase in adoption of Apache Cassandra as a leading NoSQL database for managing large data volumes, but we have also seen many clients experiencing difficulty converting their high expectations into operational Cassandra performance. Here we present a high-level technical overview of the major strengths and limitations of Cassandra that we have observed over the last few years while helping our clients resolve the real-world issues that they have experienced.

Read More Read More

Many improvements in Java 8 API for Akka

June 24, 2016 | Software Consultancy

Many improvements in Java 8 API for Akka

Akka has been designed with a Java API from the very first version. Though widely adopted, as a Java developer I think Akka has been mainly a Scala thing… until recently. Things are changing and Akka is moving to a proper Java 8 support.

Read More Read More

The Seven Deadly Sins of Project Managers

May 27, 2016 | Software Consultancy

The Seven Deadly Sins of Project Managers

People make mistakes. People can improve. People need to be told of their mistakes. Surprisingly project managers are people too… I know… in some cases it is really hard to believe it; and then they can improve too! Shocking! I have more than 15 years working in IT, occupying different roles from software development to project management in three different countries. I’ve seen really bad project managers and some really good project managers; Yes, they DO exist!

Read More Read More

The Concursus Programming Model: Kotlin

April 29, 2016 | Software Consultancy

The Concursus Programming Model: Kotlin

In this post, I’ll demonstrate an alternative API which uses some of the advanced language features of the new Kotlin language from Jetbrains. As Kotlin is a JVM-based language, it interoperates seamlessly with Concursus’s Java 8 classes; however, it also offers powerful ways to extend their functionality.

Read More Read More

The Concursus Programming Model: State

April 28, 2016 | Software Consultancy

The Concursus Programming Model: State

In a conventional RDBMS-with-ORM system, we are used to thinking of domain objects as mapped to rows in database tables, and of the database as a repository where the current state of every object exists simultaneously, so that what we get when we query for an object is the state that object was in at the time the query was issued. To perform an update, we can start a transaction, retrieve the current state of the object, modify it, save it back again and commit. Transactions move the global state of the system from one consistent state to another, so that the database transaction log represents a single, linear history of updates. We are therefore able to have a very stable, intuitive sense of what it means to talk about the “current state” of any domain object.

Read More Read More

Securing Terraform state with Vault

April 2, 2016 | Terraform Provider

Securing Terraform state with Vault

When it comes to automating the creation of infrastructure in cloud providers, Terraform (version at time of writing 0.6.14) has become one of my core go to tools in this space. It provides a fantastic declarative approach to describing the resources you want, and then takes care of making it so for you, keeping track of the state in either a local file or a remote store of some sort. Various bits of sensitive data is often provided as input to terraform.

Read More Read More

Test Automation Concepts – Parallel test execution

March 14, 2016 | Software Consultancy

Test Automation Concepts – Parallel test execution

Test automation provides fast feedback on regressions. In order to achieve this tests need to execute quickly, something which becomes more of a problem as test suites grow. This is especially true of tests which exercise a user interface where the interaction with the system is slower.

A good way to address this is to have your tests execute in parallel rather than consecutively. Given sufficient resources this allows your execution time to remain low almost indefinitely as more scenarios are added to the suite.

Read More Read More

React/Redux boilerplate

March 3, 2016 | Software Consultancy

React/Redux boilerplate

In this post, I’ll be sharing some React/Redux boilerplate code that Vince Martinez and I have been developing recently. It’s primarily aimed at developers who are familiar with the React ecosystem, so if you are new to React and/or Redux, you might like to have a look at Getting Started with React and Getting Started with Redux.

Read More Read More

DevOps and IT Operations

January 29, 2016 | DevOps

DevOps and IT Operations

DevOps is 2016’s tech holy grail – unified development and operations, both working to deliver what the business needs, quickly, reliably, and adaptably. Done well, DevOps transforms the way organisations work; it helps break down barriers between tech teams, and between technology and the rest of the business. Good DevOps is the antidote to increasing segmentation and specialisation within companies. With the promised benefits, is it any wonder that senior managers are pushing for it in organisations spanning all sizes and industries?

Read More Read More

Hazelcast and Spring-managed Transactions: A Sample Integration

January 26, 2016 | Data Engineering

Hazelcast and Spring-managed Transactions: A Sample Integration

In this second post about Hazelcast and Spring, I’m integrating Hazelcast and Spring-managed transaction for a specific use case: A transactional Queue. More specifically, I want to make the message polling, of my sample chat application, transactional.

Read More Read More

Facebook’s React, and the Signal:Noise Ratio

January 15, 2016 | Software Consultancy

Facebook’s React, and the Signal:Noise Ratio

So, you’ve started to hear a lot about React, the Javascript library developed by Facebook, but is it something you need to investigate? It’s time to distil the signal from the noise, position React amongst its rivals, and provide an indication of where it currently would – and would not – be a suitable fit.

Read More Read More

RAML 1.0 promotes reusability and standardisation

November 25, 2015 | Microservices

RAML 1.0 promotes reusability and standardisation

In May a 1.0 release of RAML (RESTful API Markup Language) has been announced delivering a few much welcome additions in the RAML 1.0 specification. This major release marks an important milestone in the evolution of RAML and indicates the team behind the specification is confident this release delivers the comprehensive set of tools for developing RESTful APIs. I’ve been using RAML 0.8 for several months now and have enjoyed the simplicity and productivity it offers for designing and documenting APIs. I must say I’m quite pleased with the changes introduced in the new release and would like to review those I consider particularly useful.

Read More Read More

Reusing ansible roles with private git repos and dependency management

November 4, 2015 | Software Consultancy

Reusing ansible roles with private git repos and dependency management

Writing reusable roles for Ansible is not an easy task but one that’s worth doing. This post should walk you through the basics of writing reusable roles with dependencies backed by public and private git repositories.

Read More Read More

JavaOne: Debugging Java Applications Running in Docker

November 3, 2015 | Software Consultancy

JavaOne: Debugging Java Applications Running in Docker

My JavaOne experience was rather busy this year, what with three talks presented in a single day! The first of these talks “Debugging Java Apps in Containers: No Heavy Welding Gear Required” was delivered with my regular co-presenter Steve Poole, from IBM, and we shared our combined experiences of working with Java and Docker over the past year.

Read More Read More

JavaOne: Building a Microservice Development Ecosystem (Video)

October 31, 2015 | Microservices

JavaOne: Building a Microservice Development Ecosystem (Video)

Microservices: Some Assembly Required

Over the past few weeks I’ve been writing an OpenCredo blog series on the topic of “Building a Microservice Development Ecosystem”, but my JavaOne talk of the same title crept up on me before I managed to finish the remaining posts. I’m still planning to finish the full blog series, but in the meantime I thought it would be beneficial to share the video and slides associated with the talk, alongside some of my related thinking. I’ve been fortunate to work on several interesting microservice projects at OpenCredo, and we’re always keen to share our knowledge or offer advice, and so please do get in touch if we can help you or your organisation.

Read More Read More

“The Business Behind Microservices” – Upcoming Webinar

September 18, 2015 | Microservices

“The Business Behind Microservices” – Upcoming Webinar

Microservices: Organisation, Architectural and Operational Challenges

We’re pleased to begin our series of OpenCredo webinars with “The Business Behind Microservices”, which takes a look at the some of the business and organisational challenges that come along with the decision to implement microservices.

Read More Read More

Lambda memoization in Java 8

August 19, 2015 | Software Consultancy

Lambda memoization in Java 8

Memoization is a technique whereby we trade memory for execution speed. Suppose you have a function which

  • Is costly to execute.
  • Always returns the same output for the same input.
  • May be called many times with the same input.

In this scenario, it may make sense to “remember” the output returned for each distinct input in a hash map, and replace function execution with a lookup in the hash map.

Read More Read More

Designing a REST API with fine-grained resources, HATEOAS and HAL

August 12, 2015 | Microservices

Designing a REST API with fine-grained resources, HATEOAS and HAL

Over the last few months one of my main projects at OpenCredo has involved creating various microservices which are interacted with via REST. We’ve been working with a relatively rich domain model, which in turn has presented a lot of challenges in how to design our various resources. This blog post aims to summarise various techniques and practices which I’ve found helpful in overcoming these challenges.

Read More Read More

OpenCredo’s Agile Transformation

August 11, 2015 | DevOps

OpenCredo’s Agile Transformation

For years, OpenCredo has been working with organisations to help them introduce new technologies, and more effective development practices, to their IT teams. This has met with a great deal of success, and we have worked with a variety of companies of various sizes. During these projects, we have consistently noticed that the changes we make reach beyond IT in their impact and effects.

Read More Read More

Boot my (secure)->(gov) cloud

August 10, 2015 | Cloud, Software Consultancy

Boot my (secure)->(gov) cloud

As a company, we at OpenCredo are heavily involved in automation and devOps based work, with a keen focus on making this a seamless experience, especially in cloud based environments. We are currently working within HMRC, a UK government department to help make this a reality as part of a broader cloud broker ecosystem project. In this blog post, I look to provide some initial insight into some of the tools and techniques employed to achieve this for one particular use case namely:
With pretty much zero human intervention, bar initiating a process and providing some inputs, a development team from any location, should be able to run “something”, which, in the end, results in an isolated, secure set of fully configured VM’s being provisioned within a cloud provider (or providers) of choice.

Read More Read More

July 28, 2015 | Microservices

Documenting REST APIs – a tooling review

Recently I co-presented a talk at Goto Amsterdam on lessons learnt whilst developing with a Microservices architecture; one being the importance of defining and documenting your API contracts as early as possible in the development cycle. During the talk I mentioned a few API documentation tools that I’d used and, based on feedback and questions from attendees, I realised that this topic merited a blog post. So, the purpose of this is to introduce 5 tools which help with designing, testing and documenting APIs.

Read More Read More

A deep dive into Angular 2.0

July 8, 2015 | Software Consultancy

A deep dive into Angular 2.0

I was quite excited around autumn last year when Google started to work on a new version of Angular (Angular 2.0) which promised to revolutionise web development. There were rumours that Angular 2.0 wouldn’t be backward compatible with its predecessor, and would be written in Google’s AtScript which is a JavaScript based language on top of Microsoft’s TypeScript. The lack of backwards compatibility raised some concerns, especially for the clients that we had used Angular at. But, lets not get ahead of ourselves here….

Read More Read More

OpenCredo: First DataStax Partner in the UK to achieve Cassandra Certification
Locally Developing Micro-Services with Docker Compose

March 11, 2015 | Microservices

Locally Developing Micro-Services with Docker Compose

Docker ComposeOne of the pain points experienced with developing microservices is that it often proves too cumbersome to replicate an environment for local development. This usually means the first time an application talks to its “real” dependencies is when it gets deployed to a shared testing environment. A relatively laborious continuous integration process usually precedes this deployment, making our feedback cycle longer than we would like. In this post I describe a workflow that aims to improve that, using Docker and Docker Compose (formerly known as fig).

Read More Read More

How to Write Your Own Apache Mesos Framework?

February 16, 2015 | Software Consultancy

How to Write Your Own Apache Mesos Framework?

Apache Mesos is often explained as being a kernel for the data-centre; meaning that cluster resources (CPU, RAM, …) are tracked and offered to “user space” programs (i.e. frameworks) to do computations on the cluster.

Read More Read More

November 19, 2014 | Microservices

The 7 Deadly Sins of Microservices

Undeniably, there is a growing interest in microservices as we see more organisations, big and small, evaluating and implementing this emerging approach. Despite its apparent novelty, most concepts and principles underpinning microservices are not exactly new – they are simply proven and commonsense software development practices that now need to be applied holistically and at a wider scale, rather than at the scale of a single program or machine. These principles include separation of concerns, loose coupling, scalability and fault-tolerance.

Read More Read More

Test Automation Framework – Quick start

November 4, 2014 | Software Consultancy

Test Automation Framework – Quick start

When starting a project, teams often spend their time re-inventing the ‘automated testing wheel’.  While every project has it’s own challenges and every team it’s own needs, many things exist as common requirements of a flexible test automation framework.

This post introduces an effective Java test framework that can be used to quickly get started with test automation on a Java project.

Read More Read More

New features in Cassandra 2.0 – Lightweight Transactions on Insert

January 6, 2014 | Cassandra

New features in Cassandra 2.0 – Lightweight Transactions on Insert

The team over at Cucumber Pro recently posted a sneak peek on their blog, demonstrating some key features of their offering.

As more of a technical user of Cucumber, there isn’t much that’s new or ground-breaking for me – almost every feature is already available through your preferred IDE combined with a few plugins.

Read More Read More

New features in Cassandra 2.0 – More on Lightweight Transactions

December 2, 2013 | Cassandra

New features in Cassandra 2.0 – More on Lightweight Transactions

Perhaps the most important of Cassandra’s selling points is its completely distributed architecture and its ability to easily extend the cluster with virtually any number of nodes. Implementing a classical RDBMS-style transaction consisting of “put locks on the database, modify the data, then commit the transaction”-style operations are simply not feasible in such an architecture (i.e. that doesn’t scale well).

Read More Read More

New features in Cassandra 2.0 – Lightweight Transactions and Triggers

November 14, 2013 | Cassandra

New features in Cassandra 2.0 – Lightweight Transactions and Triggers

Cassandra 2.0 was released in early September this year and came with some interesting new features, including “lightweight transactions” and triggers.

Despite the rising interest in the various non-relational databases in recent years, there are still numerous use-cases for which a relational database system is a better choice. The latest major release of Cassandra (version 2.0) provides some interesting features that aim to close this gap, and offers its fast and distributed storage engine enhanced with new options that will make users’ lives easier.

Read More Read More

Cross platform test automation with Appium, WebDriver and Cucumber-JVM

September 19, 2013 | Software Consultancy

Cross platform test automation with Appium, WebDriver and Cucumber-JVM

This post will give an overview of mobile testing using Appium. We will integrate tests for a native Android application into an existing Cucumber-JVM based set of acceptance tests and demonstrate multi platform testing from a single set of BDD scenarios. The sample code for this can be found here.

Read More Read More

Running Cucumber JVM tests in parallel

July 2, 2013 | Software Consultancy

Running Cucumber JVM tests in parallel

This blog post will address the issue of slow test runs when using Cucumber JVM with web automation tools such as WebDriver to perform acceptance testing on a web application.

If your team is using continuous integration this becomes especially noticeable, forcing teams to either wait for acceptance tests to complete before deploying or having to ignore the bulk of the tests.

The sample code and description in this post will show you how to convert your suite to running tests in parallel, something which has historically been problematic with unanswered questions and outstanding Cucumber-JVM bugs on the subject. Tying the described approach in with Selenium Grid2 will allow you distribute your testing across several machines if your suite is especially large or slow.

Read More Read More

Deploying Neo4j Graph Database Server across AWS regions with Puppet

August 16, 2012 | Neo4j

Deploying Neo4j Graph Database Server across AWS regions with Puppet

neo4jIt’s been more than a year now since I rolled out Neo4j Graph Database Server image in Amazon EC2.

In May 2011 the version of Neo4j was 1.3 and just recently guys at Neo Technology published version 1.7.2 so I thought now is the time to revisit this exercise and make fresh AMIs available.
Last year I created Neo4j AMI manually in one region then copied it across to the remaining AWS regions. Due to the size of the AMI and the latency between regions this process was slow.

Read More Read More

Esper Extensions – Implementing Custom Aggregation Function

March 21, 2012 | Software Consultancy

Esper Extensions – Implementing Custom Aggregation Function

Event processing Language (EPL) enables us to write complex queries to get the most out of our event stream in real time, using SQL-like syntax.

EPL allows us to use full power of aggregation of the high volume event stream to get required results with the minimal latency. In this blog we are going to explore some aspects of numerical aggregation of data with high precision BigDecimal values. We will also demonstrate how you can add you own aggregation function to Esper engine and use them in EPL statements.

Read More Read More

A Simple Introduction to Complex Event Processing – Stock Ticker End-to-End Sample

February 8, 2012 | Data Analysis, Data Engineering

A Simple Introduction to Complex Event Processing – Stock Ticker End-to-End Sample

Most of the important players in this space are large IT corporations like Oracle and IBM with their commercial (read expensive) offerings.

While most of CEP products offer some great features, it’s license model and close code policy doesn’t allow developers to play with them on pet projects, which would drive adoption and usage of CEP in every day programming.

Read More Read More

Practical behaviour driven development – Maintainable Acceptance Tests

February 3, 2012 | Software Consultancy

Practical behaviour driven development – Maintainable Acceptance Tests

Behaviour driven development (BBD)

Creating maintainable acceptance test

In this post we will look at the creation of a suite of maintainable acceptance tests, identifying some key issues as encountered on real projects and suggesting solutions.This post assumes some knowledge of behaviour driven development (BDD). If you are new to this concept, I’d recommend Dan North’s blog as a starting point, particularly his post introducing the concept of BDD.The idea of human readable acceptance testing championed by BDD enthusiasts is one that continues to grow in popularity, and it’s easy to see why.

Read More Read More