Open Credo

123 items found: Search results for "api" in all categories x

Many improvements in Java 8 API for Akka

June 24, 2016 | Software Consultancy

Many improvements in Java 8 API for Akka

Akka has been designed with a Java API from the very first version. Though widely adopted, as a Java developer I think Akka has been mainly a Scala thing… until recently. Things are changing and Akka is moving to a proper Java 8 support.

Read More Read More

Easy API Simulation with Hoverfly JUnit Rule

March 17, 2016 | Software Consultancy

Easy API Simulation with Hoverfly JUnit Rule

In order to be able to regularly release an application, your automated tests must be set up to give you fast and reliable feedback loop. If bugs are only found during a long and expensive multi-service or end-to-end test run, it can be a hinderance to fast delivery. Unfortunately I have often seen this problem in development environments: a huge suite of clunky, flaky and slow end-to-end tests which test the full functionality of the application as opposed to being more lightweight and reflecting basic user journeys. This produces the “ice cream cone” anti-pattern of test coverage, where unit tests aren’t providing the kind of coverage and feedback they need to.

Read More Read More

Implementing HAL hypermedia REST API using Spring HATEOAS

November 1, 2015 | Microservices

Implementing HAL hypermedia REST API using Spring HATEOAS

To use or not to use hypermedia (HATEOAS) in a REST API, to attain the Level 3 of the famous Richardson Maturity Model. This is one of the most discussed subjects about API design.
The many objections make sense (“Why I hate HATEOAS“, “More objections to HATEOAS“…)
. The goal of having fully dynamic, auto-discovering clients is still unrealistic (…waiting for AI client libraries).

However, there are good examples of successful HATEOAS API. Among them, PayPal.

Read More Read More

Designing a REST API with fine-grained resources, HATEOAS and HAL

August 12, 2015 | Microservices

Designing a REST API with fine-grained resources, HATEOAS and HAL

Over the last few months one of my main projects at OpenCredo has involved creating various microservices which are interacted with via REST. We’ve been working with a relatively rich domain model, which in turn has presented a lot of challenges in how to design our various resources. This blog post aims to summarise various techniques and practices which I’ve found helpful in overcoming these challenges.

Read More Read More

July 28, 2015 | Microservices

Documenting REST APIs – a tooling review

Recently I co-presented a talk at Goto Amsterdam on lessons learnt whilst developing with a Microservices architecture; one being the importance of defining and documenting your API contracts as early as possible in the development cycle. During the talk I mentioned a few API documentation tools that I’d used and, based on feedback and questions from attendees, I realised that this topic merited a blog post. So, the purpose of this is to introduce 5 tools which help with designing, testing and documenting APIs.

Read More Read More

API Testing with Cucumber

February 4, 2013 | Software Consultancy

API Testing with Cucumber

This API will in future be used by a mobile client and by third parties, making it important to verify that it is functionally correct as well as clearly documented.

An additional requirement in our case is for the tests to form a specification for the API to allow front and back end developers agree on the format in advance. This is something that BDD excels at, making it natural to continue to use Cucumber. This post will focus on the difficulties of attaining the appropriate level of abstraction with Cucumber while retaining the technical detail required for specification.

Read More Read More

When Your Product Teams Should Aim to be Inefficient – Part 2

June 29, 2022 | Blog, Organisational Transformation, Software Consultancy

When Your Product Teams Should Aim to be Inefficient – Part 2

Many businesses advocate for efficiency, but this is not always the right goal.

In part one of this article, we explored how product teams can balance two important considerations – efficiency and effectiveness.

In this second part we will introduce the – often unexpected – implications of turning to technology to bring about efficiency and wider change, and the deeper considerations that must be addressed first.

Read More Read More

Lunch & Learn: GraphQL on Springboot

March 10, 2022 | Data Engineering, Open Source

Lunch & Learn: GraphQL on Springboot

In this lunch & learn session, Ebru Cucen and Alberto Faedda explore the historical background of GraphQL with case examples and a demo of how it can be used.

Read More Read More

DZone Repost: Testing Serverless Functions

February 11, 2022 | AWS, Cloud, GCP, Kubernetes, Microservices, Open Source, Software Consultancy

DZone Repost: Testing Serverless Functions

Serverless functions are easy to install and upload, but we can’t ignore the basics. This article looks at different strategies related to testing serverless functions.

Read More Read More

What you might have missed in Kubernetes 1.22 release

December 5, 2021 | Cloud, Kubernetes

What you might have missed in Kubernetes 1.22 release

Kubernetes’ second release in 2021, version 1.22, has been out for a little while now and with 1.23 on its way, we thought we’d take a look back. Kubernetes 1.22 was a highly comprehensive release with 53 enhancements in all three graduation levels: 13 features have graduated to stable, 24 enhancements reached beta status, and 16 new features have been accepted into the alpha stage. 

The latest version has some noteworthy security features such as running Kubelet without root access, pod security policies, and seccomp. There are also a couple of deprecated and removed APIs. In this blog, we’ll discuss the significant changes in v1.22, as well as how to handle the removed APIs.

Read More Read More

Exploring How Policy-as-Code and OPA Fit into the K8s World

November 4, 2021 | Kubernetes

Exploring How Policy-as-Code and OPA Fit into the K8s World

We always read that ‘security is everyone’s responsibility’. For any organisation, big or small, security should always be the primary concern—not a mere afterthought. In terms of Kubernetes, securing a cluster is challenging because it has so many moving parts and, apart from securing our Kubernetes environment, we also want to control what an end-user can do in our cluster.

To achieve these goals, we can start with the built-in features provided by Kubernetes like Role-Based Access Control (RBAC), Network Policies, Secrets Management, and Pod Security Policies (PSP). But we know these features are not enough. For example, we may want specific policies like ‘all pods must have specific labels’. And even if we have the policies in place, the next big question is how to enforce them on our Kubernetes cluster in an easy and repeatable manner.

In this blog post, we’ll address this challenge and other questions pertaining to OPA and how it can integrate into Kubernetes.

Read More Read More

Running the Cloud from your Kubernetes Cluster

September 2, 2021 | Blog, Cloud, Kubernetes

Running the Cloud from your Kubernetes Cluster

In this blog, Stuart compares the new approach of deploying cloud resources as Kubernetes custom resources rather than the (now) typical approach using Terraform – or cloud specific: CloudFormation (AWS), Deployment Manager (GCP). He also identifies what resources are suitable for this approach and which ones are not.

Read More Read More

Kafka vs RabbitMQ: The Consumer-Driven Choice

July 20, 2021 | Blog, Data Engineering, Kafka

Kafka vs RabbitMQ: The Consumer-Driven Choice

Message and event-driven systems provide an array of benefits to organisations of all shapes and sizes. At their core, they help decouple producers and consumers so that each can work at their own pace without having to wait for the other – asynchronous processing at its best.

In fact, such systems enable a whole range of messaging patterns, offering varying levels of guarantees surrounding the processing and consumption options for clients. Take for example the publish/subscribe pattern, which enables one message to be broadcast and consumed by multiple consumers; or the competing consumer pattern, which enables a message to be processed once but with multiple concurrent consumers vying for the honour—essentially providing a way to distribute the load. The manner in which these patterns are actually realised however, depends a lot on the technology used, as each has its own approach and unique tradeoffs. 

In this article we will explore how this all applies to RabbitMQ and Apache Kafka, and how these two technologies differ, specifically from a message consumer’s perspective.

Read More Read More

Machine Learning at scale: first impressions of Kubeflow

April 20, 2021 | Data Engineering, Machine Learning, Software Consultancy

Machine Learning at scale: first impressions of Kubeflow

Our recent client was a Fintech who had ambitions to build a Machine Learning platform for real-time decision making. The client had significant Kubernetes proficiency, ran on the cloud, and had a strong preference for using free, open-source software over cloud-native offerings that come with lock-in. Several components were spiked with success (feature preparation with Apache Beam and Seldon for model serving performed particularly strongly). Kubeflow was one of the next technologies on our list of spikes, showing significant promise at the research stage and seemingly a good match for our client’s priorities and skills.

That platform slipped down the client’s priority list before completing the research for Kubeflow, so I wanted to see how that project might have turned out. Would Kubeflow have made the cut?

 

Read More Read More

Anthos – A Holistic Approach to your Hybrid Cloud initiative

February 17, 2021 | Blog, Cloud, Cloud Native, GCP, Open Source

Anthos – A Holistic Approach to your Hybrid Cloud initiative

Multi-cloud is rapidly becoming the cloud strategy of choice for enterprises looking to modernise their applications.

And the reason is simple – it gives them much more flexibility to host their workloads and data where it suits them best.

In this post, we focus on Google’s application modernisation solution Google Anthos and the role it can play in your cloud transformation strategy.

Read More Read More

WebAssembly – Where is it going?

December 11, 2020 | Cloud, Cloud Native, Kubernetes, Microservices

WebAssembly – Where is it going?

“WebAssembly is a safe, portable, low-level code format designed for efficient execution and compact representation.” – W3C

In this blog, I’ll cover the different applications of Wasm and WASI, some of the projects that are making headway, and the implications for modern architectures and distributed systems.

Read More Read More

Hacking Kubernetes on AWS (EKS) from a Mac

October 29, 2020 | Cloud, Kubernetes, Open Source

Hacking Kubernetes on AWS (EKS) from a Mac

While working with a client recently, we experienced some issues when attempting to make use of NLB external load balancer services when using AWS EKS. I wanted to investigate whether these issues had been fixed in the upstream GitHub Kubernetes repository, or if I could fix it myself, contributing back to the community in the process.

Read More Read More

Decision time with AWS Keyspaces

September 22, 2020 | AWS, Blog, Cassandra, Cloud, DevOps, Open Source

Decision time with AWS Keyspaces

With the upcoming Cassandra 4.0 release, there is a lot to look forward to. Most excitingly, and following a refreshing realignment of the Open Source community around Cassandra, the next release promises to focus on fundamentals: stability, repair, observability, performance and scaling.

We must set this against the fact that Cassandra ranks pretty highly in the Stack Overflow most dreaded databases list and the reality that Cassandra is expensive to configure, operate and maintain. Finding people who have the prerequisite skills to do so is challenging.

Read More Read More

Automation of complex IT systems

May 14, 2020 | Blog, DevOps

Automation of complex IT systems

At the time of this post, the UK is making steps to exit from an unprecedented lockdown measures for the Coronavirus. Much of the UK workforce are still making efforts to work-from-home with mainly key workers operating – at risk – in public. Many industries have shut down completely. Consequently, many businesses are reflecting on what happens next and how do we better mitigate future pandemic events?

Read More Read More

It’s 2020, can I have my ML models now?

April 2, 2020 | Machine Learning

It’s 2020, can I have my ML models now?

Recent years have seen many companies consolidate all their data into a data lake/warehouse of some sort. Once it’s all consolidated, what next?

Many companies consolidate data with a field of dreams mindset – “build it and they will come”, however a comprehensive data strategy is needed if the ultimate goals of an organisation are to be realised: monetisation through Machine Learning and AI is an oft-cited goal. Unfortunately, before one rushes into the enticing world of machine learning, one should lay more mundane foundations. Indeed, in data science, estimates vary between 50% to 80% of the time taken is devoted to so-called data-wrangling. Further, Google estimates ML projects produce 5% ML code and 95% “glue code”. If this is the reality we face, what foundations are required before one can dive headlong into ML?

Read More Read More

Security, Usability & Cloud Data Services in Finance

March 20, 2020

Security, Usability & Cloud Data Services in Finance

Traditionally, Usability and Security have been set in opposition to each other: with tight security, we end up with painful user experience. In this blog, Guy focuses on financial services as an exemplar of how we can introduce usability into a vertical with challenging security and compliance requirements.

Read More Read More

Evolutionary challenges faced by VC funded organisations

November 13, 2019 | Software Consultancy

Evolutionary challenges faced by VC funded organisations

Pioneering and pushing technology boundaries – pretty much a given nowadays for the software-driven startup. Here are some insights we’ve observed working with a number of venture capital (VC) companies who have managed to navigate the choppy waters and successfully grow their business including winning further investment along the way.

With our deep hands-on technical expertise and pragmatic focus, OpenCredo has become a natural software acceleration partner for VC funded organisations who are looking to deliver tangible value as effectively as possible. We’ve been brought in to work alongside these innovators at various stages of their journey. As such we’ve gained an appreciation for and acquired, first-hand insight into some of the pressures and challenges faced. From getting and securing that next round of funding, to grappling with the technical decisions and challenges inherent in sensibly evolving offerings to accommodate future growth and scaling.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Evolving Your Architecture Whilst Still Keeping The Lights On

September 12, 2019 | Cloud Native, Microservices, Software Consultancy

Evolving Your Architecture Whilst Still Keeping The Lights On

As a technology leader, you’ll be aware that competitive pressures and shifting business requirements are driving changes in the technical architectures of many organisations. This means you need a new strategic approach based on the ability to continually evolve elements of your systems and architectures.

Read More Read More

Kafka Connect – Source Connectors: A detailed guide to connecting to what you love.

July 30, 2019 | Blog, Kafka

Kafka Connect – Source Connectors: A detailed guide to connecting to what you love.

Writing your own Kafka source connectors with Kafka Connect. In this blog, Rufus takes you on a code walk, through the Gold Verified Venafi Connector while pointing out the common pitfalls

Read More Read More

Securing Kafka using Vault PKI

February 20, 2019 | DevOps, Hashicorp, Kafka, Open Source

Securing Kafka using Vault PKI

Creating and managing a Public Key Infrastructure (PKI) could be a very straightforward task if you use appropriate tools. In this blog post, I’ll cover the steps to easily set up a PKI with Vault from HashiCorp, and use it to secure a Kafka Cluster.

Read More Read More

HA Prometheus – The Thanos Evolution

February 5, 2019 | Cloud Native, DevOps, Kubernetes, Microservices, Open Source

HA Prometheus – The Thanos Evolution

While Prometheus has fast become the standard for monitoring in the cloud, making Prometheus highly available can be tricky. This blog post will walk you through how to do this using the open source tool Thanos.

Read More Read More

Data Science on Steroids: Productionised Machine Learning as a Value Driver for Business

July 31, 2018 | Machine Learning

Data Science on Steroids: Productionised Machine Learning as a Value Driver for Business

Machine Learning, alongside a mature Data Science, will help to bring IT and business closer together. By leveraging data for actionable insights, IT will increasingly drive business value. Agile and DevOps practices enable the continuous delivery of business value through productionised machine learning models and software delivery.

Read More Read More

Devoxx UK 2018 (video): Multi Env Deploys with Terraform as Application Glue with Ben Vickers
Heuristics for Identifying Service Boundaries

May 16, 2018 | Microservices

Heuristics for Identifying Service Boundaries

To identify service boundaries, it is not enough to consider (business) domains only. Other forces like organisational communication structures, and – very important – time, strongly suggest that we should include several other criteria in our considerations.

Read More Read More

[Past event] Devoxx UK 2018

Join us for Devoxx UK 2018! We are excited to announce that we are sponsoring and speaking at Devoxx UK 2018. Devoxx is a conference by developers, for developers. Conference Tracks include: Cloud, Containers & Infratructure Big Data & AI Architecture Server Side Java Modern Web Programming Languages Security and much more… Multi Env Deploys […]

View All Past Events Read More

Microservices Anti-patterns: It’s All About the People

April 18, 2018 | Microservices

Microservices Anti-patterns: It’s All About the People

Quite a few of the anti-patterns we observe today on microservices projects are strongly related to how people approach the problem. Given their nature, these anti-patterns tend to be deeply ingrained and self-sustaining. Addressing them starts with increased awareness and by changing ways of approaching the problem, rather than by the introduction of yet another technical tool or framework.

Read More Read More

Events and Commands: Two Faces of the Same Coin?

March 14, 2018 | Data Engineering, Microservices

Events and Commands: Two Faces of the Same Coin?

Events are obviously the fundamental building block of event-sourced systems. Commands are equally a common concept in such systems although the distinction between events and commands, if any, is not always clear. There are certainly varying views on what role each one should play.

Read More Read More

Get your -aas in gear: Privatelink and PaaS democratisation on AWS

February 6, 2018 | Cloud

Get your -aas in gear: Privatelink and PaaS democratisation on AWS

Among the many announcements made at Re:Invent 2017 was the release of AWS Privatelink for Customer and Partner services. We believe that the opportunity signalled by this modest announcement may have an impact far broader than first impressions suggest.

Read More Read More

Writing a custom JupyterHub Spawner

January 11, 2018 | Data Engineering

Writing a custom JupyterHub Spawner

The last few years have seen Python emerge as a lingua franca for data scientists. Alongside Python we have also witnessed the rise of Jupyter Notebooks, which are now considered a de facto data science productivity tool, especially in the Python community. Jupyter Notebooks started as a university side-project known as iPython in circa 2001 at UC Berkeley.

Read More Read More

[Past event] Applied Data Engineering Meetup #2

We’re back with our second meetup with lightning talks from Tareq Abedrabbo, CTO of OpenCredo, David Dawson, Software Engineer and Systems Architect and Allison Wells, Data Engineer of Kobalt Music followed by discussion over beer and pizza!

View All Past Events Read More

[Past event] Muon User Group #1

We are excited to be hosting the first Muon User Group at our London office for an Intro to Muon – How to build Polyglot Message and Event Microservices!

View All Past Events Read More

Terraform Provider Development

August 9, 2017 | Cloud, DevOps, Terraform Provider

Terraform Provider Development

The recent 0.10.0 release of HashiCorp Terraform, saw a significant change to the way Providers are managed. Specifically, the single open source code repository for Terraform has been divided into core and multiple provider repositories.

 

Read More Read More

Riak, the Dynamo paper and life beyond Basho

August 8, 2017 | Cassandra

Riak, the Dynamo paper and life beyond Basho

Recently, the sad news has emerged that Basho, which developed the Riak distributed database, has gone into receivership. This would appear to present a problem for those who have adopted the commercial version of the Riak database (Riak KV) supported by Basho.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

OpenCredo Cloud Report: July 2017

July 11, 2017 | Cloud, Cloud Native

OpenCredo Cloud Report: July 2017

Over the years, OpenCredo’s projects have become increasingly tied to the public cloud. Our skills in delivering cloud infrastructure and cloud native applications have deepened and the range of cloud projects we are able to take on has grown. From enterprise cloud brokers to cloud platform migration in restricted compliance environments, our ability to deliver on the cloud is now a core component of our value proposition.

Read More Read More

CockroachDB: First Impressions

June 15, 2017 | Data Engineering

CockroachDB: First Impressions

CockroachDB is a distributed SQL (“NewSQL”) database developed by Cockroach Labs and has recently reached a major milestone: the first production-ready, 1.0 release. We at OpenCredo have been following the progress of CockroachDB for a while, and we think it’s a technology of great potential to become the go-to solution for a having a general-purpose database in the cloud.

Read More Read More

[Past event] Webinar: Detecting stolen AWS credential usage with Apache Spark

Join us as we conclude our recent Apache Spark series with a webinar that will explore the use case of “Detecting stolen AWS credential usage with Spark”

View All Past Events Read More

New Blog Series: Spark – The Pragmatic Bits

April 25, 2017 | Cassandra, Data Analysis, Data Engineering

New Blog Series: Spark – The Pragmatic Bits

Apache Spark is a powerful open source processing engine which is fast becoming our technology of choice for data analytic projects here at OpenCredo. For many years now we have been helping our clients to practically implement and take advantage of various big data technologies including the like of Apache Cassandra amongst others.

 

 

Read More Read More

Data Analytics using Cassandra and Spark

March 23, 2017 | Cassandra, Data Analysis, Data Engineering

Data Analytics using Cassandra and Spark

In recent years, Cassandra has become one of the most widely used NoSQL databases: many of our clients use Cassandra for a variety of different purposes. This is no accident as it is a great datastore with nice scalability and performance characteristics.

However, adopting Cassandra as a single, one size fits all database has several downsides. The partitioned/distributed data storage model makes it difficult (and often very inefficient) to do certain types of queries or data analytics that are much more straightforward in a relational database.

Read More Read More

Automating Your Security Acceptance Tests

March 23, 2017 | Data Engineering, Machine Learning

Automating Your Security Acceptance Tests

On previous blog posts we have provided examples of different types of acceptance tests coverage, UI, API and Performance. One area where automation is often lacking is around validating the security of the application under test. This has been discussed in the post on non functional testing You Are Ignoring Non-functional Testing. With this post we will enhance the automation framework to quickly check for some common security flaws.

Read More Read More

Programmable Infrastructure Needs Testing Too

March 20, 2017 | DevOps

Programmable Infrastructure Needs Testing Too

DevOps has swept the tech landscape. Now, many are discovering the benefits of programmable infrastructure. I have been lucky to work on many projects where we’ve taken advantage of tools such as Terraform, Ansible, or Chef.

 

Read More Read More

Google Cloud Spanner: our first impressions

March 7, 2017 | Data Analysis, GCP

Google Cloud Spanner: our first impressions

Google has recently made its internal Spanner database available to the wider public, as a hosted solution on Google Cloud. This is a distributed relational/transactional database used inside for various Google projects (including F1, the advertising backend), promising high throughput, low latency and 99.999% availability. As such it is an interesting alternative to many open source or other hosted solutions. This whitepaper gives a good theoretical introduction into Spanner.

Read More Read More

Some Uses For Type Aliases in Kotlin 1.1

February 28, 2017 | Software Consultancy

Some Uses For Type Aliases in Kotlin 1.1

As Kotlin’s 1.1 release draws closer, I’ve been looking at some of the new language features it supports. Type aliases may seem like a relatively minor feature next to coroutines, but as I will show in this blog post, they can open up a new programming idiom, particularly when combined with extension functions.

Read More Read More

Event Replaying with Hazelcast Jet

February 13, 2017 | Data Engineering

Event Replaying with Hazelcast Jet

One of the stated intentions behind the design of Java 8’s Streams API was to take better advantage of the multi-core processing power of modern computers. Operations that could be performed on a single, linear stream of values could also be run in parallel by splitting that stream into multiple sub-streams, and combining the results from processing each sub-stream as they became available.

Read More Read More

[Past event] OOP Konferenz 2017

Join Andrew Morgan  and Daniel  Bryant at OOP Conference 2017! By combining software and business, OOP Conference 2017 is the meeting point for people who work primarily in the enterprise environment. Technical experts (architects, developers, business analysts, and testers), technical (project) managers and leaders gain an excellent view on the state-of-the-art in modern software engineering. […]

View All Past Events Read More

[Past event] Microservices Meetup Munich – Deadly Sins and Effective Testing

Our Chief Scientist, Daniel Bryant and Consultant Andrew Morgan will be heading to Munich for the Microservices Meetup on the 1st of Feb. The night will kick off at 6:30pm at Einstein Kultur with drinks and pizza, talks from Daniel and Andrew on microservices and effective testing will then follow from 7pm.

View All Past Events Read More

Reactive event processing with Reactor Core: a first look

January 26, 2017 | Data Engineering

Reactive event processing with Reactor Core: a first look

Suppose you are given the task of writing code that fulfils the following contract:

  • You will be given a promise that, at some point in the future, some data – a series of values – will become available.
  • In return, you will supply a promise that, at some point in the future, some data representing the results of processing that data will become available.
  • There may be more values to process than you can fit in memory, or even an infinite series of values.
  • You are allowed to specify what will be done with each individual value, as and when it becomes available; this includes discarding some values.
  • Whenever you want to use some external service to do something with a value, that service can only return you a promise that, at some point in the future, some data representing the result of processing that value will become available.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

[Past event] Mesos User Group #14

The Mesos User Group is back on this month and will be hosted by Capita. Come and join us for a great night and hear talks by Darren Davison and Crispin Brown!

View All Past Events Read More

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

January 24, 2017 | Cloud

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

This blog aims to provide an end to end example of how you can automatically request, generate and install a free HTTPS/TLS/SSL certificate from Let’s Encrypt using Terraform. Let’s Encrypt is a free, automated, and open certificate authority (CA) aiming to make it super easy (and free – did I say free!) for people to obtain HTTPS (SSL/TLS) certificates for their websites and infrastructure. Under the hood, Let’s Encrypt implements and leverages an emerging protocol called ACME to make all this magic happen, and it is this ACME protocol that powers the Terraform provider we will be using. For more information on how Let’s Encrypt and the ACME protocol actually work, please see how Let’s Encrypt works.

Read More Read More

[Past event] London Mesos User Group #13

The London Mesos User Group is back for its monthly meetup on October the 20th! We are excited to announce that a couple of MesosCon EU speakers have kindly agreed to join us and would like thank GoCardless who are providing us with a venue and JFrog for very kindly supplying the beer and pizza. […]

View All Past Events Read More

From Java to Go, and Back Again

October 13, 2016 | Data Analysis

From Java to Go, and Back Again

In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program. As you’re writing a program you may think “I wish Lisp had such-and-such an operator.” So you go and write it. Afterward you realize that using the new operator would simplify the design of another part of the program, and so on. Language and program evolve together…In the end your program will look as if the language had been designed for it. And when language and program fit one another well, you end up with code which is clear, small, and efficient – Paul Graham, Programming Bottom-Up

Read More Read More

[Past event] Jax London 2016

Join Daniel Bryant and Andrew Morgan at Jax London 2016 this year on the 11th of October!

View All Past Events Read More

Simulating GitHub OAuth2 Login for Tests with JUnit, Webdriver & Hoverfly: Record, Sanitise, Playback and Assert

September 21, 2016 | DevOps

Simulating GitHub OAuth2 Login for Tests with JUnit, Webdriver & Hoverfly: Record, Sanitise, Playback and Assert

Sometimes, it can be difficult to write automated tests for parts of your application due to complexities introduced by an external dependency. It may be flaky or have some sort of rate limiting, or require sensitive information which we don’t want to expose outside of our production environment. To get around this, teams might take the approach of manually stubbing the service or using mocks – but the former is tedious and error prone, whereas the latter doesn’t test collaboration at all.

Read More Read More

Kubernetes from scratch to AWS with Terraform and Ansible (part 3)

August 26, 2016 | Kubernetes

Kubernetes from scratch to AWS with Terraform and Ansible (part 3)

This post is the last of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible. To understand the goal of the project, you’d better start from the first part.

Read More Read More

Kubernetes from scratch to AWS with Terraform and Ansible (part 2)

August 26, 2016 | Kubernetes

Kubernetes from scratch to AWS with Terraform and Ansible (part 2)

This post is the second of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible. To understand the goal of the project, you’d better start from the first part.

Read More Read More

Kubernetes from scratch to AWS with Terraform and Ansible (part 1)

August 26, 2016 | Kubernetes

Kubernetes from scratch to AWS with Terraform and Ansible (part 1)

This post is the first of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible.

Read More Read More

[Past event] Microservices Manchester

OpenCredo is going North with Microservices Manchester! We are super excited to announce ourselves as Event Partners and organisers of Microservices Manchester. It will be held on the 5th of July 2016 and is a free single day two-track conference. Microservices Manchester is specifically designed for technical implementers and decision makers who wish to better understand […]

View All Past Events Read More

Key Takeaways from the DevOps Enterprise Summit (#DOES16) EU Conference

July 3, 2016 | DevOps

Key Takeaways from the DevOps Enterprise Summit (#DOES16) EU Conference

Several of us from the OpenCredo team were in attendance at the inaugural EU edition of the DevOps Enterprise Summit conference. We have been big fans of the two previous US versions, and have watched the video recordings of talks (2014, 2015) with keen interest as many of our DevOps transformation clients are very much operating in the ‘enterprise’ space.

Read More Read More

The Destructor Pattern

June 3, 2016 | Software Consultancy

The Destructor Pattern

Complexity warning

In this post, I’m going to take something extremely simple, unfold it into something disconcertingly complex, and then fold it back into something relatively simple again. The exercise isn’t entirely empty: in the process, we’ll derive a more powerful (because more generic) version of the extremely simple thing we started with. I’m describing the overall shape of the journey now, because programmers who don’t love complexity for its own sake often find the initial “unfolding” stage objectionable, and then have trouble regarding the eventual increase in fanciness as worth the struggle.

Read More Read More

Introducing KubeFuse: A File System for Kubernetes

May 31, 2016 | Kubernetes

Introducing KubeFuse: A File System for Kubernetes

Do you ever wake up and think to yourself: oh geez, Kubernetes is awesome, but I wish I could browse and edit my services and replication controllers using the file system? No? Well, in any case, this is now possible.

Read More Read More

The Concursus Programming Model: Kotlin

April 29, 2016 | Software Consultancy

The Concursus Programming Model: Kotlin

In this post, I’ll demonstrate an alternative API which uses some of the advanced language features of the new Kotlin language from Jetbrains. As Kotlin is a JVM-based language, it interoperates seamlessly with Concursus’s Java 8 classes; however, it also offers powerful ways to extend their functionality.

Read More Read More

The Concursus Programming Model: Under the Hood

April 29, 2016 | Software Consultancy

The Concursus Programming Model: Under the Hood

In the previous two posts (part 1, and part 2), we looked at how Concursus uses method mapping to generate events from method calls on proxies, and to dispatch events to matching methods on event handlers and state class instances. This approach provides a concise, convenient client API to the Concursus event system; however the core of the system defines events and event-handling mechanisms without reference to any of the reflection-based machinery used to implement this API. It is perfectly possible (if comparatively cumbersome) to use a Concursus event store to read and write events without using reflection. In this post I’ll show how this is done, continuing with the “lightbulb” example introduced previously.

Read More Read More

Test Automation Concepts – Test Data and Aliases

April 5, 2016 | Software Consultancy

Test Automation Concepts – Test Data and Aliases

This post is part of a series which introduce key concepts in successful test automation. Each post contains sample code using the test-automation-quickstart project, a sample Java test automation framework available from Github.

Read More Read More

Test Automation Concepts – Automated email testing

March 29, 2016 | Software Consultancy

Test Automation Concepts – Automated email testing

Raise your test coverage with automated email testing

Acceptance test suites generally are used for UI and API testing, and we have covered both these approaches in our Test Automation Quickstart project. However, an application may, for example, send registration or expiration warning emails. Often, tests related to this are left to manual testing, instead of putting them into an automated test suite.

However, there’s no need to check emails manually: it suffers from all the same problems as other manual testing. It’s slow, expensive, and inconsistent. There are many libraries available to interact with email through code – this post will focus on how to use them within an automated test suite.

Read More Read More

Product Development Is Science

March 7, 2016 | DevOps

Product Development Is Science

Businesses exist to make money; their purpose isn’t just to generate revenue, but to create profits, now and in the future. Generating profits means delivering products or services that people want to buy. The creation of what people want is the entire purpose of delivery pipelines. (NB: The rest of this article will use ‘product’ to refer to both products and services.)

Read More Read More

Kotlin: a new JVM language you should try

March 3, 2016 | Software Consultancy

Kotlin: a new JVM language you should try

JetBrains (the people behind IntelliJ IDEA) have recently announced  the first RC for version 1.0 of Kotlin, a new programming language for the JVM. I say ‘new’, but Kotlin has been in the making for a few years now, and has been used by JetBrains to develop several of their products, including Intellij IDEA. The company open-sourced Kotlin in 2011, and have worked with the community since then to make the language what it is today.

Read More Read More

Is it Time for Your ‘Microservices Checkup’?

March 2, 2016 | DevOps, Microservices

Is it Time for Your ‘Microservices Checkup’?

Many of our clients are currently implementing applications using a ‘microservice’-based architecture. Increasingly we are hearing from organisations that are part way through a migration to microservices, and they want our help with validating and improving their current solution. These ‘microservices checkup’ projects have revealed some interesting patterns, and because we have experience of working in a wide-range of industries (and also have ‘fresh eyes’ when looking at a project), we are often able to work alongside teams to make significant improvements and create a strategic roadmap for future improvements.

Read More Read More

ContainerSched 2016: The Container & Scheduler Conference – Call for Papers
Hazelcast and Spring-managed Transactions: A Sample Integration

January 26, 2016 | Data Engineering

Hazelcast and Spring-managed Transactions: A Sample Integration

In this second post about Hazelcast and Spring, I’m integrating Hazelcast and Spring-managed transaction for a specific use case: A transactional Queue. More specifically, I want to make the message polling, of my sample chat application, transactional.

Read More Read More

Akka Typed brings type safety to Akka framework

January 18, 2016 | Software Consultancy

Akka Typed brings type safety to Akka framework

Last time in this series I summarised all the Akka Persistence related improvements in Akka 2.4. Since then Akka 2.4.1 has been released with some additional bug fixes and improvements so perhaps now is a perfect time to pick up this mini-series and introduce some other new features included in Akka 2.4.x.

Read More Read More

The Seven Deadly Sins of Microservices (Redux)

January 8, 2016 | Microservices

The Seven Deadly Sins of Microservices (Redux)

Many of our clients are in the process of investigating or implementing ‘microservices’, and a popular question we often get asked is “what’s the most common mistake you see when moving towards a microservice architecture?”. We’ve seen plenty of good things with this architectural pattern, but we have also seen a few recurring issues and anti-patterns, which I’m keen to share here.

Read More Read More

Improving Flow

January 7, 2016 | DevOps, Software Consultancy

Improving Flow

Good consulting is, by its nature, an act of collaboration. We recently helped a company with a variety of challenges – some architecture, some coding, some systems, some people, some process (normal consultancy challenges) – unique to this client. During the project, we formalised some things we had thought before, but which had never crystallised – all the work we did was transformative. Whether it’s a code review, process review, DevOps implementation, or outright transformation, the primary goal is the same – improving flow. Flow (sometimes known as throughput) is the movement of raw materials through a system to become finished goods. It’s analogy in the service industry is the movement of customer requirements through to usable solution. And we help improve it.

Read More Read More

[Past event] Microservices for the Enterprise: A webinar by ZeroTurnaround

Join our Chief Scientist, Daniel Bryant who will be Co-presenting on the ‘Microservices for the Enterprise’ webinar by ZeroTurnaround!

View All Past Events Read More

CloudCredo acquired by Pivotal

[Past event] µCon 2015

Join us at µCon 2015, the conference on Microservices, REST, dddesign & SOA. Network, share and learn with world leading architects and microservices experts.

View All Past Events Read More

(Spring) Booting Hazelcast

December 1, 2015 | Software Consultancy

(Spring) Booting Hazelcast

This post introduce some of the basic features of Hazelcast, some of its limitations, how to embed it in a Spring Boot application and write integration testings. This post is intended to be the first of a series about Hazelcast and its integration with Spring (Boot). Let’s start from the basics.

Read More Read More

RAML 1.0 promotes reusability and standardisation

November 25, 2015 | Microservices

RAML 1.0 promotes reusability and standardisation

In May a 1.0 release of RAML (RESTful API Markup Language) has been announced delivering a few much welcome additions in the RAML 1.0 specification. This major release marks an important milestone in the evolution of RAML and indicates the team behind the specification is confident this release delivers the comprehensive set of tools for developing RESTful APIs. I’ve been using RAML 0.8 for several months now and have enjoyed the simplicity and productivity it offers for designing and documenting APIs. I must say I’m quite pleased with the changes introduced in the new release and would like to review those I consider particularly useful.

Read More Read More

Webinar invite: Deploying HashiCorp Vault for secure cloud deployments
JavaOne: Debugging Java Applications Running in Docker

November 3, 2015 | Software Consultancy

JavaOne: Debugging Java Applications Running in Docker

My JavaOne experience was rather busy this year, what with three talks presented in a single day! The first of these talks “Debugging Java Apps in Containers: No Heavy Welding Gear Required” was delivered with my regular co-presenter Steve Poole, from IBM, and we shared our combined experiences of working with Java and Docker over the past year.

Read More Read More

JavaOne: Building a Microservice Development Ecosystem (Video)

October 31, 2015 | Microservices

JavaOne: Building a Microservice Development Ecosystem (Video)

Microservices: Some Assembly Required

Over the past few weeks I’ve been writing an OpenCredo blog series on the topic of “Building a Microservice Development Ecosystem”, but my JavaOne talk of the same title crept up on me before I managed to finish the remaining posts. I’m still planning to finish the full blog series, but in the meantime I thought it would be beneficial to share the video and slides associated with the talk, alongside some of my related thinking. I’ve been fortunate to work on several interesting microservice projects at OpenCredo, and we’re always keen to share our knowledge or offer advice, and so please do get in touch if we can help you or your organisation.

Read More Read More

What’s new in Akka Persistence 2.4

October 28, 2015 | Software Consultancy

What’s new in Akka Persistence 2.4

Let’s have a quick look at the most interesting changes and new features that are now available to Akka users. As there are many new features to highlight in the new Akka release I will focus on those related to Akka Persistence first and cover other areas in a separate post.

Read More Read More

Join OpenCredo at JavaOne in San Francisco

October 18, 2015 | DevOps, Microservices

Join OpenCredo at JavaOne in San Francisco

Microservices, Debugging Containers and Software Development Methodologies

Once again I’m privileged to be speaking at the premier Java conference, JavaOne in San Francisco. This year I will be presenting (at least) three conferences sessions: “Building a Microservice Ecosystem”, “Debugging Java Apps in Containers” and “Thinking, Fast and Slow, with Software Development”. I say ‘at least’ three talks as I usually get press-ganged volunteered into helping out at other talks and BoF sessions, but this is simply a sign of the great community spirit and a large group of friends involved with this conference!

Read More Read More

DevOps is Transformative

October 12, 2015 | DevOps

DevOps is Transformative

The Pre-DevOps Environment

DevOps is transformative. This (hopefully) won’t be true forever, but it is for now. While the modern management practices of separating development and operations (and to a lesser extent, everyone else) prevail, the tearing down of the walls that separate them will remain transformative. In company after company, management and front-line staff are coming to realise that keeping functions separate, which are inherently interdependent, is a model for blame, shifted responsibility, and acrimony. It’s easy to divvy-up a company up based on function. To many people, it seems the most logical way to do it. Ops does operations, Dev does development, Marketing markets, etc. It seems much harder to do it any other way. So why do it?

Read More Read More

The Business Behind Microservices Webinar (Video and Slides)

September 24, 2015 | Microservices

The Business Behind Microservices Webinar (Video and Slides)

Unless you’ve been living under a (COBOL-based) rock for the last few years, you will have no doubt heard of the emerging trend of microservices. This approach to developing ‘loosely coupled service-oriented architecture with bounded contexts’ has captured the hearts and minds of many developers. The promise of easier enforcement of good architectural and design principles, such as encapsulation and interface segregation, combined with the availability to experiment with different languages and platforms for each service, is a (developer) match made in heaven.

Read More Read More

Microservice Platforms: Some Assembly [Still] Required. Part Two

September 20, 2015 | Microservices

Microservice Platforms: Some Assembly [Still] Required. Part Two

Working Locally with Microservices

Over the past five years I have worked within several projects that used a ‘microservice’-based architecture, and one constant issue I have encountered is the absence of standardised patterns for local development and ‘off the shelf’ development tooling that support this. When working with monoliths we have become quite adept at streamlining the development, build, test and deploy cycles. Development tooling to help with these processes is also readily available (and often integrated with our IDEs). For example, many platforms provide ‘hot reloading’ for viewing the effects of code changes in near-real time, automated execution of tests, regular local feedback from continuous integration servers, and tooling to enable the creation of a local environment that mimics the production stack.

Read More Read More

Terraform Infrastructure Design Patterns

September 14, 2015 | Cloud, DevOps

Terraform Infrastructure Design Patterns

If you are operating in the programmable infrastructure space, you will hopefully have come across Terraform, a tool from HashiCorp which is primarily used to manage infrastructure resources such as virtual machines, DNS names and firewall settings across a number of public and private providers (AWS, GCP, Azure, …).

Read More Read More

OpenCredo’s Agile Transformation

August 11, 2015 | DevOps

OpenCredo’s Agile Transformation

For years, OpenCredo has been working with organisations to help them introduce new technologies, and more effective development practices, to their IT teams. This has met with a great deal of success, and we have worked with a variety of companies of various sizes. During these projects, we have consistently noticed that the changes we make reach beyond IT in their impact and effects.

Read More Read More

Boot my (secure)->(gov) cloud

August 10, 2015 | Cloud, Software Consultancy

Boot my (secure)->(gov) cloud

As a company, we at OpenCredo are heavily involved in automation and devOps based work, with a keen focus on making this a seamless experience, especially in cloud based environments. We are currently working within HMRC, a UK government department to help make this a reality as part of a broader cloud broker ecosystem project. In this blog post, I look to provide some initial insight into some of the tools and techniques employed to achieve this for one particular use case namely:
With pretty much zero human intervention, bar initiating a process and providing some inputs, a development team from any location, should be able to run “something”, which, in the end, results in an isolated, secure set of fully configured VM’s being provisioned within a cloud provider (or providers) of choice.

Read More Read More

GOTO London 2015 “Agile, Lean and Rugged”
Building a Google analytics dashboard with Python3, Tornado and deploying it on OpenShift (for free)

August 5, 2015 | Data Analysis, Data Engineering

Building a Google analytics dashboard with Python3, Tornado and deploying it on OpenShift (for free)

A few weeks ago, we thought about building a Google analytics dashboard to give us easy access to certain elements of our Google Analytics web traffic. We saw some custom dashboards for bloggers, but nothing quite right for our goal, since we wanted the data on a big screen for everyone in the office to view.

Read More Read More

Embracing Disruptive Innovation: OpenCredo Partners with Google

August 5, 2015 | Cloud, GCP, Kubernetes

Embracing Disruptive Innovation: OpenCredo Partners with Google

Why OpenCredo partnered with Google

Recently OpenCredo chose to partner with Google in order to share knowledge and resources around the Google Cloud Platform offerings. Our clients come in many shapes and sizes, but typically all of them realise three disruptive truths of the modern IT industry: the (economic) value of cloud; the competitive advantage of continuous delivery; and the potential of hypothesis and data-driven product development to increase innovation (as popularised by the Lean Startup / Lean Enterprise motto of ‘build, measure, learn’).

Read More Read More

New Tricks with Dynamic Proxies in Java 8 (part 1)

July 13, 2015 | Software Consultancy

New Tricks with Dynamic Proxies in Java 8 (part 1)

Why Use Dynamic Proxies?

Dynamic proxies have been a feature of Java since version 1.3. They were widely used in J2EE for remoting. Given an abstract interface, and a concrete implementation of that interface, a call to some method on the interface can be made “remote” (i.e. cross-JVM) by creating two additional classes. The first, a “marshalling” implementation of the interface, captures the details of the call in the source JVM and serializes them over the network. The second, an “unmarshalling” endpoint, receives the serialized call details and dispatches the call to an instance of the concrete class on the target JVM.

Read More Read More

A deep dive into Angular 2.0

July 8, 2015 | Software Consultancy

A deep dive into Angular 2.0

I was quite excited around autumn last year when Google started to work on a new version of Angular (Angular 2.0) which promised to revolutionise web development. There were rumours that Angular 2.0 wouldn’t be backward compatible with its predecessor, and would be written in Google’s AtScript which is a JavaScript based language on top of Microsoft’s TypeScript. The lack of backwards compatibility raised some concerns, especially for the clients that we had used Angular at. But, lets not get ahead of ourselves here….

Read More Read More

Asynchronous Cloud bootstrapping with Terraform, Cloud-Init & Puppet

June 23, 2015 | Cloud, DevOps, Terraform Provider

Asynchronous Cloud bootstrapping with Terraform, Cloud-Init & Puppet

Working with OpenCredo clients, I’ve noticed that even if you are one of the few organisations that can boast ‘Infrastructure as Code’, perhaps it’s only true of your VMs, and likely you have ‘bootstrap problems’. What I mean by this, is that you require some cloud-infrastructure to already be in place before your VM automation can go to work.

Read More Read More

OpenCredo and Container Solutions Partner to Deliver Emerging Technologies
Supporting the G-Cloud 6 framework …
How to Write Your Own Apache Mesos Framework?

February 16, 2015 | Software Consultancy

How to Write Your Own Apache Mesos Framework?

Apache Mesos is often explained as being a kernel for the data-centre; meaning that cluster resources (CPU, RAM, …) are tracked and offered to “user space” programs (i.e. frameworks) to do computations on the cluster.

Read More Read More

OpenCredo is pleased to announce the publication of Neo4J In Action
Visit us at QCon London 2015

November 19, 2014 | Microservices

The 7 Deadly Sins of Microservices

Undeniably, there is a growing interest in microservices as we see more organisations, big and small, evaluating and implementing this emerging approach. Despite its apparent novelty, most concepts and principles underpinning microservices are not exactly new – they are simply proven and commonsense software development practices that now need to be applied holistically and at a wider scale, rather than at the scale of a single program or machine. These principles include separation of concerns, loose coupling, scalability and fault-tolerance.

Read More Read More

Test Automation Framework – Quick start

November 4, 2014 | Software Consultancy

Test Automation Framework – Quick start

When starting a project, teams often spend their time re-inventing the ‘automated testing wheel’.  While every project has it’s own challenges and every team it’s own needs, many things exist as common requirements of a flexible test automation framework.

This post introduces an effective Java test framework that can be used to quickly get started with test automation on a Java project.

Read More Read More

October 23, 2014 | Cassandra

Spring Data Cassandra Overview

Spring Data Cassandra (SDC) is a community project under the Spring Data (SD) umbrella that provides convenient and familiar APIs to work with Apache Cassandra.

Read More Read More

Experiences with Spring Boot

February 24, 2014 | Cloud Native, Microservices

Experiences with Spring Boot

Last year some of us attended the London Spring eXchange where we encountered a new and interesting tool that Pivotal was working on: Spring Boot. Since then we had the opportunity to see what it’s capable of in a live project and we were deeply impressed.

Read More Read More

Cucumber Android vs Appium with cucumber jvm

January 28, 2014 | Software Consultancy

Cucumber Android vs Appium with cucumber jvm

A while ago I published this blog post about writing tests for mobile applications using Appium and cucumber-jvm.

In this post, I will look at an alternative approach to testing an Android native application using Cucumber-Android.

Throughout the post I will draw comparisons between Appium and Cucumber-Android, the goal being to determine the best approach for testing an android application using Cucumber. I will focus on the ease of configuration and use, speed of test runs and quality of reporting.

Read More Read More

New features in Cassandra 2.0 – Lightweight Transactions and Triggers

November 14, 2013 | Cassandra

New features in Cassandra 2.0 – Lightweight Transactions and Triggers

Cassandra 2.0 was released in early September this year and came with some interesting new features, including “lightweight transactions” and triggers.

Despite the rising interest in the various non-relational databases in recent years, there are still numerous use-cases for which a relational database system is a better choice. The latest major release of Cassandra (version 2.0) provides some interesting features that aim to close this gap, and offers its fast and distributed storage engine enhanced with new options that will make users’ lives easier.

Read More Read More

Cross platform test automation with Appium, WebDriver and Cucumber-JVM

September 19, 2013 | Software Consultancy

Cross platform test automation with Appium, WebDriver and Cucumber-JVM

This post will give an overview of mobile testing using Appium. We will integrate tests for a native Android application into an existing Cucumber-JVM based set of acceptance tests and demonstrate multi platform testing from a single set of BDD scenarios. The sample code for this can be found here.

Read More Read More

Configuration Management with Flexible Contexts

August 2, 2013 | Software Consultancy

Configuration Management with Flexible Contexts

Configuration management was born in the pre-cloud era. Remember the days when acquiring a super powerful multi core server felt like winning the jackpot? Infrastructure was a slightly different place back then. Yet for all the recent developments in DevOps, its legacy is still with us.

Read More Read More

Spring Data Hadoop – Contextual Analysis

July 4, 2013 | Software Consultancy

Spring Data Hadoop – Contextual Analysis

In which situations Spring Data Hadoop (SDH) can add value, and in which situations would it be a poor choice? This article follows on from an objective summary of the features of SDH.

Read More Read More

Running Cucumber JVM tests in parallel

July 2, 2013 | Software Consultancy

Running Cucumber JVM tests in parallel

This blog post will address the issue of slow test runs when using Cucumber JVM with web automation tools such as WebDriver to perform acceptance testing on a web application.

If your team is using continuous integration this becomes especially noticeable, forcing teams to either wait for acceptance tests to complete before deploying or having to ignore the bulk of the tests.

The sample code and description in this post will show you how to convert your suite to running tests in parallel, something which has historically been problematic with unanswered questions and outstanding Cucumber-JVM bugs on the subject. Tying the described approach in with Selenium Grid2 will allow you distribute your testing across several machines if your suite is especially large or slow.

Read More Read More

Spring Data Hadoop – Objective Overview

June 30, 2013 | Software Consultancy

Spring Data Hadoop – Objective Overview

Spring Data Hadoop (SDH) is a Spring offshoot project that allows the invocation and configuration of Hadoop tasks within a Spring application context. It offers support for Hadoop jobs, HBase, Pig, Hive, Cascading and additionally JSR-223 scripting for job preparation and tidy-up.

It is most suited for use in organisations with existing Spring applications or investment in Spring expertise. Some SDH features replicate functionality of tools in the Hadoop ecosystem that DevOps engineers who maintain a Hadoop cluster will be more familiar with.

 

Read More Read More

Model Matters: Graphs, Neo4j and the Future

February 25, 2013 | Neo4j

Model Matters: Graphs, Neo4j and the Future

As part of our work, we often help our customers choose the right datastore for a project. There are usually a number of considerations involved in that process, such as performance, scalability, the expected size of the data set, and the suitability of the data model to the problem at hand.

This blog post is about my experience with graph database technologies, specifically Neo4j. I would like to share some thoughts on when Neo4j is a good fit but also what challenges Neo4j faces now and in the near future.

Read More Read More

Withstanding the test of time

December 18, 2012 | Software Consultancy

Withstanding the test of time

The first thing most people think of when they start a project with the good intentions of test driven development is: write a test first. That’s great, and something I would fully encourage. However, diving in to writing tests without forethought, especially on large projects with a lot of developers can lead to new problems that TDD is not going to solve. With some upfront thinking (but not big upfront design!) a large team can avoid problems later down the line by considering some important and desirable traits of a large and rapidly changing test suite.

Read More Read More

Deploying Lift Applications to Cloud Foundry

February 15, 2012 | Cloud

Deploying Lift Applications to Cloud Foundry

As part of a recent project, I have been working on a number of Scala/Lift applications that we are hosting on a private Cloud Foundry instance.

In this blog post I would like to talk about some practical aspects of developing and deploying Lift applications to Cloud Foundry.

Out of the box, Cloud Foundry is able to run simple Lift applications smoothly. Things however become more interesting if your application needs to talk to one of the available services on Cloud Foundry, such as a relational or a NoSql store.

Read More Read More

A Simple Introduction to Complex Event Processing – Stock Ticker End-to-End Sample

February 8, 2012 | Data Analysis, Data Engineering

A Simple Introduction to Complex Event Processing – Stock Ticker End-to-End Sample

Most of the important players in this space are large IT corporations like Oracle and IBM with their commercial (read expensive) offerings.

While most of CEP products offer some great features, it’s license model and close code policy doesn’t allow developers to play with them on pet projects, which would drive adoption and usage of CEP in every day programming.

Read More Read More

Practical behaviour driven development – Maintainable Acceptance Tests

February 3, 2012 | Software Consultancy

Practical behaviour driven development – Maintainable Acceptance Tests

Behaviour driven development (BBD)

Creating maintainable acceptance test

In this post we will look at the creation of a suite of maintainable acceptance tests, identifying some key issues as encountered on real projects and suggesting solutions.This post assumes some knowledge of behaviour driven development (BDD). If you are new to this concept, I’d recommend Dan North’s blog as a starting point, particularly his post introducing the concept of BDD.The idea of human readable acceptance testing championed by BDD enthusiasts is one that continues to grow in popularity, and it’s easy to see why.

Read More Read More

Neo4j Super Nodes and Indexed Relationships, Part II

November 4, 2011 | Neo4j

Neo4j Super Nodes and Indexed Relationships, Part II

 Neo4j Super Nodes

In the previous post we compared the performance of fetching relationships from densely populated nodes using Neo4j native store and using lucene index.

We’ve seen that we can fetch the small subset of relationships from a super-node (containing ~1M relationships in total) directly from the Lucene index, the performance of the first run (cold-caches) is better then using the Neo store directly. The subsequent runs with caches warmed up show comparable performance, slightly in favor of direct Neo store fetching, sue to low level cache optimizations.

Read More Read More

Neo4j: Super-Nodes and Indexed Relationships, Part I

June 3, 2011 | Neo4j

Neo4j: Super-Nodes and Indexed Relationships, Part I

Neo4J is one of the first graph databases to appear on the global market. Being open source, in addition to its power and simplicity in supporting graph data model it represents good choice for production-ready graph database.

However, there has been one area I have struggled to get good-enough performance from Neo4j recently – super nodes.

Read More Read More

OpenCredo is Host for QCon London’s “Enterprise Agile Transformation” Track