Open Credo

30 items found: Search results for "decision making" in all categories x

Machine Learning at scale: first impressions of Kubeflow

April 20, 2021 | Data Engineering, Machine Learning, Software Consultancy

Machine Learning at scale: first impressions of Kubeflow

Our recent client was a Fintech who had ambitions to build a Machine Learning platform for real-time decision making. The client had significant Kubernetes proficiency, ran on the cloud, and had a strong preference for using free, open-source software over cloud-native offerings that come with lock-in. Several components were spiked with success (feature preparation with Apache Beam and Seldon for model serving performed particularly strongly). Kubeflow was one of the next technologies on our list of spikes, showing significant promise at the research stage and seemingly a good match for our client’s priorities and skills.

That platform slipped down the client’s priority list before completing the research for Kubeflow, so I wanted to see how that project might have turned out. Would Kubeflow have made the cut?

 

Read More Read More

Improving Flow

January 7, 2016 | DevOps, Software Consultancy

Improving Flow

Good consulting is, by its nature, an act of collaboration. We recently helped a company with a variety of challenges – some architecture, some coding, some systems, some people, some process (normal consultancy challenges) – unique to this client. During the project, we formalised some things we had thought before, but which had never crystallised – all the work we did was transformative. Whether it’s a code review, process review, DevOps implementation, or outright transformation, the primary goal is the same – improving flow. Flow (sometimes known as throughput) is the movement of raw materials through a system to become finished goods. It’s analogy in the service industry is the movement of customer requirements through to usable solution. And we help improve it.

Read More Read More

Software Circus: Thinking Fast and Slow with Software Development

September 13, 2015 | DevOps

Software Circus: Thinking Fast and Slow with Software Development

Making Good Decisions within Software

Last week I was privileged to be able to present my “Thinking Fast and Slow with Software Development” talk at the inaugural Software Circus conference in Amsterdam. The conference was amazing, and I’ll write more about this later, but in this post I was keen to share the presentation slides and the thinking behind this talk…

Read More Read More

Exploring How Policy-as-Code and OPA Fit into the K8s World

November 4, 2021 | Kubernetes

Exploring How Policy-as-Code and OPA Fit into the K8s World

We always read that ‘security is everyone’s responsibility’. For any organisation, big or small, security should always be the primary concern—not a mere afterthought. In terms of Kubernetes, securing a cluster is challenging because it has so many moving parts and, apart from securing our Kubernetes environment, we also want to control what an end-user can do in our cluster.

To achieve these goals, we can start with the built-in features provided by Kubernetes like Role-Based Access Control (RBAC), Network Policies, Secrets Management, and Pod Security Policies (PSP). But we know these features are not enough. For example, we may want specific policies like ‘all pods must have specific labels’. And even if we have the policies in place, the next big question is how to enforce them on our Kubernetes cluster in an easy and repeatable manner.

In this blog post, we’ll address this challenge and other questions pertaining to OPA and how it can integrate into Kubernetes.

Read More Read More

Automation of complex IT systems

May 14, 2020 | Blog, DevOps

Automation of complex IT systems

At the time of this post, the UK is making steps to exit from an unprecedented lockdown measures for the Coronavirus. Much of the UK workforce are still making efforts to work-from-home with mainly key workers operating – at risk – in public. Many industries have shut down completely. Consequently, many businesses are reflecting on what happens next and how do we better mitigate future pandemic events?

Read More Read More

It’s 2020, can I have my ML models now?

April 2, 2020 | Machine Learning

It’s 2020, can I have my ML models now?

Recent years have seen many companies consolidate all their data into a data lake/warehouse of some sort. Once it’s all consolidated, what next?

Many companies consolidate data with a field of dreams mindset – “build it and they will come”, however a comprehensive data strategy is needed if the ultimate goals of an organisation are to be realised: monetisation through Machine Learning and AI is an oft-cited goal. Unfortunately, before one rushes into the enticing world of machine learning, one should lay more mundane foundations. Indeed, in data science, estimates vary between 50% to 80% of the time taken is devoted to so-called data-wrangling. Further, Google estimates ML projects produce 5% ML code and 95% “glue code”. If this is the reality we face, what foundations are required before one can dive headlong into ML?

Read More Read More

Security, Usability & Cloud Data Services in Finance

March 20, 2020

Security, Usability & Cloud Data Services in Finance

Traditionally, Usability and Security have been set in opposition to each other: with tight security, we end up with painful user experience. In this blog, Guy focuses on financial services as an exemplar of how we can introduce usability into a vertical with challenging security and compliance requirements.

Read More Read More

[Past event] O’Reilly Software Architecture 2018

We are excited to announce our Senior Consultant, Fahran Wallace is speaking this year at O’Reilly Software Architecture 2018.

View All Past Events Read More

Heuristics for Identifying Service Boundaries

May 16, 2018 | Microservices

Heuristics for Identifying Service Boundaries

To identify service boundaries, it is not enough to consider (business) domains only. Other forces like organisational communication structures, and – very important – time, strongly suggest that we should include several other criteria in our considerations.

Read More Read More

A Pragmatic Introduction to Machine Learning for DevOps Engineers

January 23, 2018 | Data Engineering, DevOps

A Pragmatic Introduction to Machine Learning for DevOps Engineers

Machine Learning is a hot topic these days, as can be seen from search trends. It was the success of Deepmind and AlphaGo in 2016 that really brought machine learning to the attention of the wider community and the world at large.

Read More Read More

[Past event] Agile Cambridge 2017

Agile Cambridge is a three day practical agile development conference that allows participants to connect and learn from their peers and leaders in the industry. Join Matt Long, our senior QA consultant on the 29th of September for the third day of Agile Cambridge for a case study on a ‘from the trenches’ experience.

View All Past Events Read More

Riak, the Dynamo paper and life beyond Basho

August 8, 2017 | Cassandra

Riak, the Dynamo paper and life beyond Basho

Recently, the sad news has emerged that Basho, which developed the Riak distributed database, has gone into receivership. This would appear to present a problem for those who have adopted the commercial version of the Riak database (Riak KV) supported by Basho.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

CockroachDB: First Impressions

June 15, 2017 | Data Engineering

CockroachDB: First Impressions

CockroachDB is a distributed SQL (“NewSQL”) database developed by Cockroach Labs and has recently reached a major milestone: the first production-ready, 1.0 release. We at OpenCredo have been following the progress of CockroachDB for a while, and we think it’s a technology of great potential to become the go-to solution for a having a general-purpose database in the cloud.

Read More Read More

Testing a Spark Application

May 9, 2017 | Cassandra

Testing a Spark Application

Data analytics isn’t a field commonly associated with testing, but there’s no reason we can’t treat it like any other application. Data analytics services are often deployed in production, and production services should be properly tested. This post covers some basic approaches for the testing of Cassandra/Spark code. There will be some code examples, but the focus is on how to structure your code to ensure it is testable!

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

Read More Read More

Google Cloud Spanner: our first impressions

March 7, 2017 | Data Analysis, GCP

Google Cloud Spanner: our first impressions

Google has recently made its internal Spanner database available to the wider public, as a hosted solution on Google Cloud. This is a distributed relational/transactional database used inside for various Google projects (including F1, the advertising backend), promising high throughput, low latency and 99.999% availability. As such it is an interesting alternative to many open source or other hosted solutions. This whitepaper gives a good theoretical introduction into Spanner.

Read More Read More

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

January 24, 2017 | Cloud

Let’s Encrypt and Terraform – Getting free certificates for your infrastructure

This blog aims to provide an end to end example of how you can automatically request, generate and install a free HTTPS/TLS/SSL certificate from Let’s Encrypt using Terraform. Let’s Encrypt is a free, automated, and open certificate authority (CA) aiming to make it super easy (and free – did I say free!) for people to obtain HTTPS (SSL/TLS) certificates for their websites and infrastructure. Under the hood, Let’s Encrypt implements and leverages an emerging protocol called ACME to make all this magic happen, and it is this ACME protocol that powers the Terraform provider we will be using. For more information on how Let’s Encrypt and the ACME protocol actually work, please see how Let’s Encrypt works.

Read More Read More

How Not To Use Cassandra Like An RDBMS (and what will happen if you do)

September 15, 2016 | Cassandra

How Not To Use Cassandra Like An RDBMS (and what will happen if you do)

Cassandra isn’t a relational database management system, but it has some features that make it look a bit like one. Chief among these is CQL, a query language with an SQL-like syntax. CQL isn’t a bad thing in itself – in fact it’s very convenient – but it can be misleading since it gives developers the illusion that they are working with a familiar data model, when things are really very different under the hood.

Read More Read More

Patterns of Successful Cassandra Data Modelling

September 6, 2016 | Cassandra

Patterns of Successful Cassandra Data Modelling

A growing number of clients are asking OpenCredo for help with using Apache Cassandra and solving specific problems they encounter. Clients have different use cases, requirements, implementation and teams but experience similar issues. We have noticed that Cassandra data modelling problems are the most consistent cause of Cassandra failing to meet their expectations. Data modelling is one of the most complex areas of using Cassandra and has many considerations.

Read More Read More

Hazelcast and Spring-managed Transactions: A Sample Integration

January 26, 2016 | Data Engineering

Hazelcast and Spring-managed Transactions: A Sample Integration

In this second post about Hazelcast and Spring, I’m integrating Hazelcast and Spring-managed transaction for a specific use case: A transactional Queue. More specifically, I want to make the message polling, of my sample chat application, transactional.

Read More Read More

Reusing ansible roles with private git repos and dependency management

November 4, 2015 | Software Consultancy

Reusing ansible roles with private git repos and dependency management

Writing reusable roles for Ansible is not an easy task but one that’s worth doing. This post should walk you through the basics of writing reusable roles with dependencies backed by public and private git repositories.

Read More Read More

Join OpenCredo at JavaOne in San Francisco

October 18, 2015 | DevOps, Microservices

Join OpenCredo at JavaOne in San Francisco

Microservices, Debugging Containers and Software Development Methodologies

Once again I’m privileged to be speaking at the premier Java conference, JavaOne in San Francisco. This year I will be presenting (at least) three conferences sessions: “Building a Microservice Ecosystem”, “Debugging Java Apps in Containers” and “Thinking, Fast and Slow, with Software Development”. I say ‘at least’ three talks as I usually get press-ganged volunteered into helping out at other talks and BoF sessions, but this is simply a sign of the great community spirit and a large group of friends involved with this conference!

Read More Read More

The Business Behind Microservices Webinar (Video and Slides)

September 24, 2015 | Microservices

The Business Behind Microservices Webinar (Video and Slides)

Unless you’ve been living under a (COBOL-based) rock for the last few years, you will have no doubt heard of the emerging trend of microservices. This approach to developing ‘loosely coupled service-oriented architecture with bounded contexts’ has captured the hearts and minds of many developers. The promise of easier enforcement of good architectural and design principles, such as encapsulation and interface segregation, combined with the availability to experiment with different languages and platforms for each service, is a (developer) match made in heaven.

Read More Read More

Boot my (secure)->(gov) cloud

August 10, 2015 | Cloud, Software Consultancy

Boot my (secure)->(gov) cloud

As a company, we at OpenCredo are heavily involved in automation and devOps based work, with a keen focus on making this a seamless experience, especially in cloud based environments. We are currently working within HMRC, a UK government department to help make this a reality as part of a broader cloud broker ecosystem project. In this blog post, I look to provide some initial insight into some of the tools and techniques employed to achieve this for one particular use case namely:
With pretty much zero human intervention, bar initiating a process and providing some inputs, a development team from any location, should be able to run “something”, which, in the end, results in an isolated, secure set of fully configured VM’s being provisioned within a cloud provider (or providers) of choice.

Read More Read More

New Tricks with Dynamic Proxies in Java 8 (part 1)

July 13, 2015 | Software Consultancy

New Tricks with Dynamic Proxies in Java 8 (part 1)

Why Use Dynamic Proxies?

Dynamic proxies have been a feature of Java since version 1.3. They were widely used in J2EE for remoting. Given an abstract interface, and a concrete implementation of that interface, a call to some method on the interface can be made “remote” (i.e. cross-JVM) by creating two additional classes. The first, a “marshalling” implementation of the interface, captures the details of the call in the source JVM and serializes them over the network. The second, an “unmarshalling” endpoint, receives the serialized call details and dispatches the call to an instance of the concrete class on the target JVM.

Read More Read More

What’s the point of Test Automation?

November 21, 2014 | White Paper

What’s the point of Test Automation?

In our latest white paper we cover the high level fundamentals of test automation, discuss the primary problems test automation solves and take a look at the steps required to implement this within your development process – all aimed at improving software delivery.

Read More Read More

OpenCredo Partners with EsperTech to Offer Solutions for High Performance, Event-Driven Systems