Open Credo

68 items found: Search results for "basic" in all categories x

Making Sense of Data with RDF* vs. LPG

January 31, 2022 | Blog, Data Engineering

Making Sense of Data with RDF* vs. LPG

There are two camps of Graph database, one side is RDF, where they are strict with their format, and somewhat limited for their extensibility. The other side is LPG, where they can define labels to the relationships. With its recent extension, RDF now allows users to add properties, thus becoming RDF*. In this blog, Ebru explores the structural and performance differences between LPG and RDF*.

Read More Read More

Kafka vs RabbitMQ: The Consumer-Driven Choice

July 20, 2021 | Blog, Data Engineering, Kafka

Kafka vs RabbitMQ: The Consumer-Driven Choice

Message and event-driven systems provide an array of benefits to organisations of all shapes and sizes. At their core, they help decouple producers and consumers so that each can work at their own pace without having to wait for the other – asynchronous processing at its best.

In fact, such systems enable a whole range of messaging patterns, offering varying levels of guarantees surrounding the processing and consumption options for clients. Take for example the publish/subscribe pattern, which enables one message to be broadcast and consumed by multiple consumers; or the competing consumer pattern, which enables a message to be processed once but with multiple concurrent consumers vying for the honour—essentially providing a way to distribute the load. The manner in which these patterns are actually realised however, depends a lot on the technology used, as each has its own approach and unique tradeoffs. 

In this article we will explore how this all applies to RabbitMQ and Apache Kafka, and how these two technologies differ, specifically from a message consumer’s perspective.

Read More Read More

Hacking Kubernetes on AWS (EKS) from a Mac

October 29, 2020 | Cloud, Kubernetes, Open Source

Hacking Kubernetes on AWS (EKS) from a Mac

While working with a client recently, we experienced some issues when attempting to make use of NLB external load balancer services when using AWS EKS. I wanted to investigate whether these issues had been fixed in the upstream GitHub Kubernetes repository, or if I could fix it myself, contributing back to the community in the process.

Read More Read More

What is Continuous Verification?

October 15, 2020

What is Continuous Verification?

Continuous Verification is a term that is starting to pop up from time-to-time… but what does it mean? Well… according to Nora Jones and Casey Rosenthal, authors of O’Reilly’s Chaos Engineering books,

 

“Continuous verification (CV) is a discipline of proactive experimentation, implemented as tooling that verifies system behaviors. This stands in contrast to prior common practices in software quality assurance, which favor reactive testing, implemented as methodologies that validate known properties of software. This isn’t to say that prior common practices are invalid or should be deprecated. Alerting, testing, code reviews, monitoring, SRE practices, and the like—these are all great practices and should be encouraged”

 

Over the course of this post, we will unpack this statement: to understand what is behind it and what it might mean for your development process.

Read More Read More

It’s 2020, can I have my ML models now?

April 2, 2020 | Machine Learning

It’s 2020, can I have my ML models now?

Recent years have seen many companies consolidate all their data into a data lake/warehouse of some sort. Once it’s all consolidated, what next?

Many companies consolidate data with a field of dreams mindset – “build it and they will come”, however a comprehensive data strategy is needed if the ultimate goals of an organisation are to be realised: monetisation through Machine Learning and AI is an oft-cited goal. Unfortunately, before one rushes into the enticing world of machine learning, one should lay more mundane foundations. Indeed, in data science, estimates vary between 50% to 80% of the time taken is devoted to so-called data-wrangling. Further, Google estimates ML projects produce 5% ML code and 95% “glue code”. If this is the reality we face, what foundations are required before one can dive headlong into ML?

Read More Read More

Technology, Kindness and Bananas

August 14, 2019 | Culture, News

Technology, Kindness and Bananas

Here, in our little nook of the internet, we usually write about our experiences of emerging technologies, the tricky coding problems we’ve solved and how we have enhanced our clients’ businesses. We do this because we are very proud of this work. It truly matters.

Today is a little different. Today I’d like to share something a little more personal. Something that brings loneliness, kindness and technology together in an oxytocin-generating, slightly awkward embrace. Because hugs matter too.

Read More Read More

Kafka Connect – Source Connectors: A detailed guide to connecting to what you love.

July 30, 2019 | Blog, Kafka

Kafka Connect – Source Connectors: A detailed guide to connecting to what you love.

Writing your own Kafka source connectors with Kafka Connect. In this blog, Rufus takes you on a code walk, through the Gold Verified Venafi Connector while pointing out the common pitfalls

Read More Read More

Securing Kafka using Vault PKI

February 20, 2019 | DevOps, Hashicorp, Kafka, Open Source

Securing Kafka using Vault PKI

Creating and managing a Public Key Infrastructure (PKI) could be a very straightforward task if you use appropriate tools. In this blog post, I’ll cover the steps to easily set up a PKI with Vault from HashiCorp, and use it to secure a Kafka Cluster.

Read More Read More

HA Prometheus – The Thanos Evolution

February 5, 2019 | Cloud Native, DevOps, Kubernetes, Microservices, Open Source

HA Prometheus – The Thanos Evolution

While Prometheus has fast become the standard for monitoring in the cloud, making Prometheus highly available can be tricky. This blog post will walk you through how to do this using the open source tool Thanos.

Read More Read More

Self-testing infrastructure-as-code

May 31, 2018 | DevOps

Self-testing infrastructure-as-code

As traditional operations has embraced the concept of code, it has benefited from ideas already prevalent in developer circles such as version control. Version control brings the benefit that not only can you see what the infrastructure was, but you can also get reviews of changes by your peers before the change is made live; known to most developers as Pull Request (PR) reviews.

Read More Read More

A Pragmatic Introduction to Machine Learning for DevOps Engineers

January 23, 2018 | Data Engineering, DevOps

A Pragmatic Introduction to Machine Learning for DevOps Engineers

Machine Learning is a hot topic these days, as can be seen from search trends. It was the success of Deepmind and AlphaGo in 2016 that really brought machine learning to the attention of the wider community and the world at large.

Read More Read More

Writing a custom JupyterHub Spawner

January 11, 2018 | Data Engineering

Writing a custom JupyterHub Spawner

The last few years have seen Python emerge as a lingua franca for data scientists. Alongside Python we have also witnessed the rise of Jupyter Notebooks, which are now considered a de facto data science productivity tool, especially in the Python community. Jupyter Notebooks started as a university side-project known as iPython in circa 2001 at UC Berkeley.

Read More Read More

Q&A with Cockroach Labs – creators of CockroachDB

October 24, 2017 | Data Engineering

Q&A with Cockroach Labs – creators of CockroachDB

Cockroach Labs, the creators of CockroachDB are coming to London for the first time since their 1.0 GA Release in May 2017. They will be taking time to talk about “The Hows & Whys of a Distributed SQL Database” at the Applied Data Engineering meetup, hosted and run by us here at OpenCredo.
We have been interested in CockroachDB for a while now, including publishing our initial impressions of the release on our blog. We thought this would be the perfect time to do a bit of a Q&A before the event! I posed Raphael Poss, a core Software Engineer at Cockroach Labs a few questions.

Read More Read More

CockroachDB: First Impressions

June 15, 2017 | Data Engineering

CockroachDB: First Impressions

CockroachDB is a distributed SQL (“NewSQL”) database developed by Cockroach Labs and has recently reached a major milestone: the first production-ready, 1.0 release. We at OpenCredo have been following the progress of CockroachDB for a while, and we think it’s a technology of great potential to become the go-to solution for a having a general-purpose database in the cloud.

Read More Read More

Testing a Spark Application

May 9, 2017 | Cassandra

Testing a Spark Application

Data analytics isn’t a field commonly associated with testing, but there’s no reason we can’t treat it like any other application. Data analytics services are often deployed in production, and production services should be properly tested. This post covers some basic approaches for the testing of Cassandra/Spark code. There will be some code examples, but the focus is on how to structure your code to ensure it is testable!

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Data Analytics using Cassandra and Spark

March 23, 2017 | Cassandra, Data Analysis, Data Engineering

Data Analytics using Cassandra and Spark

In recent years, Cassandra has become one of the most widely used NoSQL databases: many of our clients use Cassandra for a variety of different purposes. This is no accident as it is a great datastore with nice scalability and performance characteristics.

However, adopting Cassandra as a single, one size fits all database has several downsides. The partitioned/distributed data storage model makes it difficult (and often very inefficient) to do certain types of queries or data analytics that are much more straightforward in a relational database.

Read More Read More

Google Cloud Spanner: our first impressions

March 7, 2017 | Data Analysis, GCP

Google Cloud Spanner: our first impressions

Google has recently made its internal Spanner database available to the wider public, as a hosted solution on Google Cloud. This is a distributed relational/transactional database used inside for various Google projects (including F1, the advertising backend), promising high throughput, low latency and 99.999% availability. As such it is an interesting alternative to many open source or other hosted solutions. This whitepaper gives a good theoretical introduction into Spanner.

Read More Read More

[Past event] Linuxing In London Meetup

We are excited to announce that we are sponsoring the Linuxing In London Meetup this month! Brett Mack, DevOps Consultant will be doing an introduction to userland rootkits, software engineer at notonthehighstreet, Nick Jackson, will be speaking about Swift coding on Linux and organiser of Linuxing In London, Brian Byrne will give a quick talk on Raspberry Pi.

View All Past Events Read More

[Past event] OOP Konferenz 2017

Join Andrew Morgan  and Daniel  Bryant at OOP Conference 2017! By combining software and business, OOP Conference 2017 is the meeting point for people who work primarily in the enterprise environment. Technical experts (architects, developers, business analysts, and testers), technical (project) managers and leaders gain an excellent view on the state-of-the-art in modern software engineering. […]

View All Past Events Read More

Reactive event processing with Reactor Core: a first look

January 26, 2017 | Data Engineering

Reactive event processing with Reactor Core: a first look

Suppose you are given the task of writing code that fulfils the following contract:

  • You will be given a promise that, at some point in the future, some data – a series of values – will become available.
  • In return, you will supply a promise that, at some point in the future, some data representing the results of processing that data will become available.
  • There may be more values to process than you can fit in memory, or even an infinite series of values.
  • You are allowed to specify what will be done with each individual value, as and when it becomes available; this includes discarding some values.
  • Whenever you want to use some external service to do something with a value, that service can only return you a promise that, at some point in the future, some data representing the result of processing that value will become available.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

January 24, 2017 | Cloud

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

This blog aims to provide an end to end example of how you can automatically request, generate and install a free HTTPS/TLS/SSL certificate from Let’s Encrypt using Terraform. Let’s Encrypt is a free, automated, and open certificate authority (CA) aiming to make it super easy (and free – did I say free!) for people to obtain HTTPS (SSL/TLS) certificates for their websites and infrastructure. Under the hood, Let’s Encrypt implements and leverages an emerging protocol called ACME to make all this magic happen, and it is this ACME protocol that powers the Terraform provider we will be using. For more information on how Let’s Encrypt and the ACME protocol actually work, please see how Let’s Encrypt works.

Read More Read More

How to TDD FizzBuzz with JUnit Theories

January 13, 2017 | Software Consultancy

How to TDD FizzBuzz with JUnit Theories

The notorious FizzBuzz interview test was originally proposed as a way of weeding out candidates for programming jobs who – to put it bluntly – couldn’t program. The task is as follows:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

It turns out that this problem has just enough subtlety about it to cause headaches to anyone who knows the basics but hasn’t learned how to think in nested structures.

Read More Read More

From Java to Go, and Back Again

October 13, 2016 | Data Analysis

From Java to Go, and Back Again

In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program. As you’re writing a program you may think “I wish Lisp had such-and-such an operator.” So you go and write it. Afterward you realize that using the new operator would simplify the design of another part of the program, and so on. Language and program evolve together…In the end your program will look as if the language had been designed for it. And when language and program fit one another well, you end up with code which is clear, small, and efficient – Paul Graham, Programming Bottom-Up

Read More Read More

[Past event] HashiCorp Meetup #6

Join Brett Mack for October’s installment of the London Hashicorp Meetup! The London Hashicorp meetup is back on the 13th of October and will be held at the Holburn Sainsbury office with the usual beers and pizza on offer! Agenda: 6.30pm – Arrive, beers socialising 7pm – Talk 1 – Brett Mack, Consultant @ OpenCredo […]

View All Past Events Read More

Simulating GitHub OAuth2 Login for Tests with JUnit, Webdriver & Hoverfly: Record, Sanitise, Playback and Assert

September 21, 2016 | DevOps

Simulating GitHub OAuth2 Login for Tests with JUnit, Webdriver & Hoverfly: Record, Sanitise, Playback and Assert

Sometimes, it can be difficult to write automated tests for parts of your application due to complexities introduced by an external dependency. It may be flaky or have some sort of rate limiting, or require sensitive information which we don’t want to expose outside of our production environment. To get around this, teams might take the approach of manually stubbing the service or using mocks – but the former is tedious and error prone, whereas the latter doesn’t test collaboration at all.

Read More Read More

Kubernetes from scratch to AWS with Terraform and Ansible (part 2)

August 26, 2016 | Kubernetes

Kubernetes from scratch to AWS with Terraform and Ansible (part 2)

This post is the second of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible. To understand the goal of the project, you’d better start from the first part.

Read More Read More

Kubernetes from scratch to AWS with Terraform and Ansible (part 1)

August 26, 2016 | Kubernetes

Kubernetes from scratch to AWS with Terraform and Ansible (part 1)

This post is the first of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible.

Read More Read More

Fulfilling the promise of Apache Cassandra performance

August 24, 2016 | Cassandra

Fulfilling the promise of Apache Cassandra performance

At OpenCredo we are seeing an increase in adoption of Apache Cassandra as a leading NoSQL database for managing large data volumes, but we have also seen many clients experiencing difficulty converting their high expectations into operational Cassandra performance. Here we present a high-level technical overview of the major strengths and limitations of Cassandra that we have observed over the last few years while helping our clients resolve the real-world issues that they have experienced.

Read More Read More

You Are Ignoring Non-­functional Testing

June 15, 2016 | Software Consultancy

You Are Ignoring Non-­functional Testing

It’s as simple as that – and as a consultant, it’s a problem I see all the time. Testing is always focused on functional testing. Non-functional testing, by comparison, is treated like a second class citizen. This means that functional requirements get refined, and non-functional requirements are ignored until the very end.

Read More Read More

Introducing KubeFuse: A File System for Kubernetes

May 31, 2016 | Kubernetes

Introducing KubeFuse: A File System for Kubernetes

Do you ever wake up and think to yourself: oh geez, Kubernetes is awesome, but I wish I could browse and edit my services and replication controllers using the file system? No? Well, in any case, this is now possible.

Read More Read More

Test Automation Concepts – Test Data and Aliases

April 5, 2016 | Software Consultancy

Test Automation Concepts – Test Data and Aliases

This post is part of a series which introduce key concepts in successful test automation. Each post contains sample code using the test-automation-quickstart project, a sample Java test automation framework available from Github.

Read More Read More

Securing Terraform state with Vault

April 2, 2016 | Terraform Provider

Securing Terraform state with Vault

When it comes to automating the creation of infrastructure in cloud providers, Terraform (version at time of writing 0.6.14) has become one of my core go to tools in this space. It provides a fantastic declarative approach to describing the resources you want, and then takes care of making it so for you, keeping track of the state in either a local file or a remote store of some sort. Various bits of sensitive data is often provided as input to terraform.

Read More Read More

Test Automation Concepts – Parallel test execution

March 14, 2016 | Software Consultancy

Test Automation Concepts – Parallel test execution

Test automation provides fast feedback on regressions. In order to achieve this tests need to execute quickly, something which becomes more of a problem as test suites grow. This is especially true of tests which exercise a user interface where the interaction with the system is slower.

A good way to address this is to have your tests execute in parallel rather than consecutively. Given sufficient resources this allows your execution time to remain low almost indefinitely as more scenarios are added to the suite.

Read More Read More

Kotlin: a new JVM language you should try

March 3, 2016 | Software Consultancy

Kotlin: a new JVM language you should try

JetBrains (the people behind IntelliJ IDEA) have recently announced  the first RC for version 1.0 of Kotlin, a new programming language for the JVM. I say ‘new’, but Kotlin has been in the making for a few years now, and has been used by JetBrains to develop several of their products, including Intellij IDEA. The company open-sourced Kotlin in 2011, and have worked with the community since then to make the language what it is today.

Read More Read More

Is it Time for Your ‘Microservices Checkup’?

March 2, 2016 | DevOps, Microservices

Is it Time for Your ‘Microservices Checkup’?

Many of our clients are currently implementing applications using a ‘microservice’-based architecture. Increasingly we are hearing from organisations that are part way through a migration to microservices, and they want our help with validating and improving their current solution. These ‘microservices checkup’ projects have revealed some interesting patterns, and because we have experience of working in a wide-range of industries (and also have ‘fresh eyes’ when looking at a project), we are often able to work alongside teams to make significant improvements and create a strategic roadmap for future improvements.

Read More Read More

Hazelcast and Spring-managed Transactions: A Sample Integration

January 26, 2016 | Data Engineering

Hazelcast and Spring-managed Transactions: A Sample Integration

In this second post about Hazelcast and Spring, I’m integrating Hazelcast and Spring-managed transaction for a specific use case: A transactional Queue. More specifically, I want to make the message polling, of my sample chat application, transactional.

Read More Read More

Akka Typed brings type safety to Akka framework

January 18, 2016 | Software Consultancy

Akka Typed brings type safety to Akka framework

Last time in this series I summarised all the Akka Persistence related improvements in Akka 2.4. Since then Akka 2.4.1 has been released with some additional bug fixes and improvements so perhaps now is a perfect time to pick up this mini-series and introduce some other new features included in Akka 2.4.x.

Read More Read More

The Seven Deadly Sins of Microservices (Redux)

January 8, 2016 | Microservices

The Seven Deadly Sins of Microservices (Redux)

Many of our clients are in the process of investigating or implementing ‘microservices’, and a popular question we often get asked is “what’s the most common mistake you see when moving towards a microservice architecture?”. We’ve seen plenty of good things with this architectural pattern, but we have also seen a few recurring issues and anti-patterns, which I’m keen to share here.

Read More Read More

[Past event] London HashiCorp User Group Meetup #3

The London HashiCorp Meetup is back on the 19th of May! We give thanks to Huddle for hosting the London HashiCorp Meetup #3, which will be kicking off at the usual time of 6.30pm with pizza and beer. Thomas Doran, Yelp’s Site Reliability Engineer will take the stage at 7pm with his talk, ‘Terraform in […]

View All Past Events Read More

(Spring) Booting Hazelcast

December 1, 2015 | Software Consultancy

(Spring) Booting Hazelcast

This post introduce some of the basic features of Hazelcast, some of its limitations, how to embed it in a Spring Boot application and write integration testings. This post is intended to be the first of a series about Hazelcast and its integration with Spring (Boot). Let’s start from the basics.

Read More Read More

RAML 1.0 promotes reusability and standardisation

November 25, 2015 | Microservices

RAML 1.0 promotes reusability and standardisation

In May a 1.0 release of RAML (RESTful API Markup Language) has been announced delivering a few much welcome additions in the RAML 1.0 specification. This major release marks an important milestone in the evolution of RAML and indicates the team behind the specification is confident this release delivers the comprehensive set of tools for developing RESTful APIs. I’ve been using RAML 0.8 for several months now and have enjoyed the simplicity and productivity it offers for designing and documenting APIs. I must say I’m quite pleased with the changes introduced in the new release and would like to review those I consider particularly useful.

Read More Read More

Reusing ansible roles with private git repos and dependency management

November 4, 2015 | Software Consultancy

Reusing ansible roles with private git repos and dependency management

Writing reusable roles for Ansible is not an easy task but one that’s worth doing. This post should walk you through the basics of writing reusable roles with dependencies backed by public and private git repositories.

Read More Read More

JavaOne: Debugging Java Applications Running in Docker

November 3, 2015 | Software Consultancy

JavaOne: Debugging Java Applications Running in Docker

My JavaOne experience was rather busy this year, what with three talks presented in a single day! The first of these talks “Debugging Java Apps in Containers: No Heavy Welding Gear Required” was delivered with my regular co-presenter Steve Poole, from IBM, and we shared our combined experiences of working with Java and Docker over the past year.

Read More Read More

Implementing HAL hypermedia REST API using Spring HATEOAS

November 1, 2015 | Microservices

Implementing HAL hypermedia REST API using Spring HATEOAS

To use or not to use hypermedia (HATEOAS) in a REST API, to attain the Level 3 of the famous Richardson Maturity Model. This is one of the most discussed subjects about API design.
The many objections make sense (“Why I hate HATEOAS“, “More objections to HATEOAS“…)
. The goal of having fully dynamic, auto-discovering clients is still unrealistic (…waiting for AI client libraries).

However, there are good examples of successful HATEOAS API. Among them, PayPal.

Read More Read More

JavaOne: Building a Microservice Development Ecosystem (Video)

October 31, 2015 | Microservices

JavaOne: Building a Microservice Development Ecosystem (Video)

Microservices: Some Assembly Required

Over the past few weeks I’ve been writing an OpenCredo blog series on the topic of “Building a Microservice Development Ecosystem”, but my JavaOne talk of the same title crept up on me before I managed to finish the remaining posts. I’m still planning to finish the full blog series, but in the meantime I thought it would be beneficial to share the video and slides associated with the talk, alongside some of my related thinking. I’ve been fortunate to work on several interesting microservice projects at OpenCredo, and we’re always keen to share our knowledge or offer advice, and so please do get in touch if we can help you or your organisation.

Read More Read More

Our Thoughts on DevOps and Cloud at JAX London

October 18, 2015 | Cloud, DevOps

Our Thoughts on DevOps and Cloud at JAX London

DevOps, Cloud and Microservices: “All Hail the Developer King/Queen”

Last week Steve Poole and I were once again back at the always informative JAX London conference talking about DevOps and the Cloud. This presentation built upon our previous DevOps talk that was presented last year, and focused on the experiences that Steve and I had encountered over the last year (the slides for our 2014 “Moving to a DevOps” mode talk can be found on SlideShare, and the video on Parleys).

Read More Read More

Join us at the Inaugural ContainerSched Conference

October 16, 2015 | Software Consultancy

Join us at the Inaugural ContainerSched Conference

Interested in Containers and Schedulers?

OpenCredo is helping Skillsmatter with the organisation of the inaugural ContainerSched conference, and we were last night in attendance at CodeNode, working our way to finalising the program alongside the Skillsmatter team. I’m pleased to say that the provisional lineup looks great (speaker acceptance emails are being sent out over the next few days), and so I wanted to share the details of some of the great content we have confirmed already.

Read More Read More

Introduction to Akka Streams – Getting started

October 1, 2015 | Data Engineering

Introduction to Akka Streams – Getting started

Going reactive

Akka Streams, the new experimental module under Akka project has been finally released in July after some months of development and several milestone and RC versions. In this series I hope to gently introduce concepts from the library and demonstrate how it can be used to address real-life stream processing challenges.

Akka Streams is an implementation of the Reactive Streams specification on top of Akka toolkit that uses actor based concurrency model. Reactive Streams specification has been created by the number of companies interested in asynchronous, non-blocking, event based data processing that can span across system boundaries and technology stacks.

Read More Read More

SaltStack – Using Consul as an External Pillar Source

September 14, 2015 | DevOps, Hashicorp, Open Source

SaltStack – Using Consul as an External Pillar Source

Recently I was working on a project that was using SaltStack for configuration management and Consul for service discovery. It occurred to me that using Consul’s key/value store would be great place to store data needed for my Salt runs, but unfortunately Consul was not supported in SaltStack as an official data store at that point in time. Being an open source project however, this provided an excellent opportunity to contribute back and this blog post looks to provide some details on how this works, as well as a practical demo on how you can take advantage of Consul as an external data store.

Read More Read More

Boot my (secure)->(gov) cloud

August 10, 2015 | Cloud, Software Consultancy

Boot my (secure)->(gov) cloud

As a company, we at OpenCredo are heavily involved in automation and devOps based work, with a keen focus on making this a seamless experience, especially in cloud based environments. We are currently working within HMRC, a UK government department to help make this a reality as part of a broader cloud broker ecosystem project. In this blog post, I look to provide some initial insight into some of the tools and techniques employed to achieve this for one particular use case namely:
With pretty much zero human intervention, bar initiating a process and providing some inputs, a development team from any location, should be able to run “something”, which, in the end, results in an isolated, secure set of fully configured VM’s being provisioned within a cloud provider (or providers) of choice.

Read More Read More

Building a Google analytics dashboard with Python3, Tornado and deploying it on OpenShift (for free)

August 5, 2015 | Data Analysis, Data Engineering

Building a Google analytics dashboard with Python3, Tornado and deploying it on OpenShift (for free)

A few weeks ago, we thought about building a Google analytics dashboard to give us easy access to certain elements of our Google Analytics web traffic. We saw some custom dashboards for bloggers, but nothing quite right for our goal, since we wanted the data on a big screen for everyone in the office to view.

Read More Read More

July 28, 2015 | Microservices

Documenting REST APIs – a tooling review

Recently I co-presented a talk at Goto Amsterdam on lessons learnt whilst developing with a Microservices architecture; one being the importance of defining and documenting your API contracts as early as possible in the development cycle. During the talk I mentioned a few API documentation tools that I’d used and, based on feedback and questions from attendees, I realised that this topic merited a blog post. So, the purpose of this is to introduce 5 tools which help with designing, testing and documenting APIs.

Read More Read More

New Tricks with Dynamic Proxies in Java 8 (part 2)

July 14, 2015 | Software Consultancy

New Tricks with Dynamic Proxies in Java 8 (part 2)

Building simple proxies

In the previous post I introduced Java dynamic proxies, and sketched out a way they could be used in testing to simplify the generation of custom Hamcrest matchers. In this post, I’m going to dive into some techniques for implementing proxies in Java 8. We’ll start with a simple case, and build up towards something more complex and full-featured.

Read More Read More

A deep dive into Angular 2.0

July 8, 2015 | Software Consultancy

A deep dive into Angular 2.0

I was quite excited around autumn last year when Google started to work on a new version of Angular (Angular 2.0) which promised to revolutionise web development. There were rumours that Angular 2.0 wouldn’t be backward compatible with its predecessor, and would be written in Google’s AtScript which is a JavaScript based language on top of Microsoft’s TypeScript. The lack of backwards compatibility raised some concerns, especially for the clients that we had used Angular at. But, lets not get ahead of ourselves here….

Read More Read More

Locally Developing Micro-Services with Docker Compose

March 11, 2015 | Microservices

Locally Developing Micro-Services with Docker Compose

Docker ComposeOne of the pain points experienced with developing microservices is that it often proves too cumbersome to replicate an environment for local development. This usually means the first time an application talks to its “real” dependencies is when it gets deployed to a shared testing environment. A relatively laborious continuous integration process usually precedes this deployment, making our feedback cycle longer than we would like. In this post I describe a workflow that aims to improve that, using Docker and Docker Compose (formerly known as fig).

Read More Read More

Traits with Java 8 Default Methods

January 30, 2015 | Software Consultancy

Traits with Java 8 Default Methods

When I first started programming in Scala a few years ago, Traits was the language feature I was most excited about. Indeed, Traits give you the ability to compose and share behaviour in a clean and reusable way. In Java, we tend deal with the same concerns by grouping crosscutting behaviour in abstract base classes that are subsequently extended every time we need to access shared behaviour in part or in total.

Read More Read More

Test Automation Framework – Quick start

November 4, 2014 | Software Consultancy

Test Automation Framework – Quick start

When starting a project, teams often spend their time re-inventing the ‘automated testing wheel’.  While every project has it’s own challenges and every team it’s own needs, many things exist as common requirements of a flexible test automation framework.

This post introduces an effective Java test framework that can be used to quickly get started with test automation on a Java project.

Read More Read More

October 23, 2014 | Cassandra

Spring Data Cassandra Overview

Spring Data Cassandra (SDC) is a community project under the Spring Data (SD) umbrella that provides convenient and familiar APIs to work with Apache Cassandra.

Read More Read More

Experiences with Spring Boot

February 24, 2014 | Cloud Native, Microservices

Experiences with Spring Boot

Last year some of us attended the London Spring eXchange where we encountered a new and interesting tool that Pivotal was working on: Spring Boot. Since then we had the opportunity to see what it’s capable of in a live project and we were deeply impressed.

Read More Read More

Cucumber Android vs Appium with cucumber jvm

January 28, 2014 | Software Consultancy

Cucumber Android vs Appium with cucumber jvm

A while ago I published this blog post about writing tests for mobile applications using Appium and cucumber-jvm.

In this post, I will look at an alternative approach to testing an Android native application using Cucumber-Android.

Throughout the post I will draw comparisons between Appium and Cucumber-Android, the goal being to determine the best approach for testing an android application using Cucumber. I will focus on the ease of configuration and use, speed of test runs and quality of reporting.

Read More Read More

New features in Cassandra 2.0 – Lightweight Transactions on Insert

January 6, 2014 | Cassandra

New features in Cassandra 2.0 – Lightweight Transactions on Insert

The team over at Cucumber Pro recently posted a sneak peek on their blog, demonstrating some key features of their offering.

As more of a technical user of Cucumber, there isn’t much that’s new or ground-breaking for me – almost every feature is already available through your preferred IDE combined with a few plugins.

Read More Read More

Withstanding the test of time – Part 2

February 19, 2013 | Software Consultancy

Withstanding the test of time – Part 2

How to create robust tests for Spring based applications

This blog post continues on from Part 1 which discussed types of tests and how to create robust tests. Part 2 will examine techniques to help whip a test suite in to shape and resolve common issues that slow everything down. The approaches in this post will focus on spring based applications, but the concepts can be applied to other frameworks too.

Read More Read More

API Testing with Cucumber

February 4, 2013 | Software Consultancy

API Testing with Cucumber

This API will in future be used by a mobile client and by third parties, making it important to verify that it is functionally correct as well as clearly documented.

An additional requirement in our case is for the tests to form a specification for the API to allow front and back end developers agree on the format in advance. This is something that BDD excels at, making it natural to continue to use Cucumber. This post will focus on the difficulties of attaining the appropriate level of abstraction with Cucumber while retaining the technical detail required for specification.

Read More Read More

A dive into saltstack

January 10, 2013 | DevOps

A dive into saltstack

Recently I have started looking into SaltStack as a solution that does both config management and orchestration. It is a relatively new project started in 2011, but it has a growing fanbase among Sys Admins and DevOps Engineers. In this blog post I will look into Salt as a promising alternative, and comparing it to Puppet as a way of exploring its basic set of features.

Read More Read More

Esper Extensions – Implementing Custom Aggregation Function

March 21, 2012 | Software Consultancy

Esper Extensions – Implementing Custom Aggregation Function

Event processing Language (EPL) enables us to write complex queries to get the most out of our event stream in real time, using SQL-like syntax.

EPL allows us to use full power of aggregation of the high volume event stream to get required results with the minimal latency. In this blog we are going to explore some aspects of numerical aggregation of data with high precision BigDecimal values. We will also demonstrate how you can add you own aggregation function to Esper engine and use them in EPL statements.

Read More Read More

48 hours – The Pursuit of Proper Performance

February 13, 2012 | Software Consultancy

48 hours – The Pursuit of Proper Performance

Recently we were approached by a client to do some performance testing of the web application they had written. The budget allowed two days for this task. Ok. No problem. Yes, we can. Naturally I had one or another question though…

Read More Read More

A Simple Introduction to Complex Event Processing – Stock Ticker End-to-End Sample

February 8, 2012 | Data Analysis, Data Engineering

A Simple Introduction to Complex Event Processing – Stock Ticker End-to-End Sample

Most of the important players in this space are large IT corporations like Oracle and IBM with their commercial (read expensive) offerings.

While most of CEP products offer some great features, it’s license model and close code policy doesn’t allow developers to play with them on pet projects, which would drive adoption and usage of CEP in every day programming.

Read More Read More

Neo4j: Super-Nodes and Indexed Relationships, Part I

June 3, 2011 | Neo4j

Neo4j: Super-Nodes and Indexed Relationships, Part I

Neo4J is one of the first graph databases to appear on the global market. Being open source, in addition to its power and simplicity in supporting graph data model it represents good choice for production-ready graph database.

However, there has been one area I have struggled to get good-enough performance from Neo4j recently – super nodes.

Read More Read More