73 items found: Search results for "rest api" in all categories x
November 1, 2015 | Microservices
To use or not to use hypermedia (HATEOAS) in a REST API, to attain the Level 3 of the famous Richardson Maturity Model. This is one of the most discussed subjects about API design.
The many objections make sense (“Why I hate HATEOAS“, “More objections to HATEOAS“…). The goal of having fully dynamic, auto-discovering clients is still unrealistic (…waiting for AI client libraries).
However, there are good examples of successful HATEOAS API. Among them, PayPal.
August 12, 2015 | Microservices
Over the last few months one of my main projects at OpenCredo has involved creating various microservices which are interacted with via REST. We’ve been working with a relatively rich domain model, which in turn has presented a lot of challenges in how to design our various resources. This blog post aims to summarise various techniques and practices which I’ve found helpful in overcoming these challenges.
July 28, 2015 | Microservices
Recently I co-presented a talk at Goto Amsterdam on lessons learnt whilst developing with a Microservices architecture; one being the importance of defining and documenting your API contracts as early as possible in the development cycle. During the talk I mentioned a few API documentation tools that I’d used and, based on feedback and questions from attendees, I realised that this topic merited a blog post. So, the purpose of this is to introduce 5 tools which help with designing, testing and documenting APIs.
March 17, 2016 | Software Consultancy
In order to be able to regularly release an application, your automated tests must be set up to give you fast and reliable feedback loop. If bugs are only found during a long and expensive multi-service or end-to-end test run, it can be a hinderance to fast delivery. Unfortunately I have often seen this problem in development environments: a huge suite of clunky, flaky and slow end-to-end tests which test the full functionality of the application as opposed to being more lightweight and reflecting basic user journeys. This produces the “ice cream cone” anti-pattern of test coverage, where unit tests aren’t providing the kind of coverage and feedback they need to.
February 4, 2013 | Software Consultancy
This API will in future be used by a mobile client and by third parties, making it important to verify that it is functionally correct as well as clearly documented.
An additional requirement in our case is for the tests to form a specification for the API to allow front and back end developers agree on the format in advance. This is something that BDD excels at, making it natural to continue to use Cucumber. This post will focus on the difficulties of attaining the appropriate level of abstraction with Cucumber while retaining the technical detail required for specification.
Writing your own Kafka source connectors with Kafka Connect. In this blog, Rufus takes you on a code walk, through the Gold Verified Venafi Connector while pointing out the common pitfalls
Google has recently made its internal Spanner database available to the wider public, as a hosted solution on Google Cloud. This is a distributed relational/transactional database used inside for various Google projects (including F1, the advertising backend), promising high throughput, low latency and 99.999% availability. As such it is an interesting alternative to many open source or other hosted solutions. This whitepaper gives a good theoretical introduction into Spanner.
December 1, 2015 | Software Consultancy
This post introduce some of the basic features of Hazelcast, some of its limitations, how to embed it in a Spring Boot application and write integration testings. This post is intended to be the first of a series about Hazelcast and its integration with Spring (Boot). Let’s start from the basics.
November 25, 2015 | Microservices
In May a 1.0 release of RAML (RESTful API Markup Language) has been announced delivering a few much welcome additions in the RAML 1.0 specification. This major release marks an important milestone in the evolution of RAML and indicates the team behind the specification is confident this release delivers the comprehensive set of tools for developing RESTful APIs. I’ve been using RAML 0.8 for several months now and have enjoyed the simplicity and productivity it offers for designing and documenting APIs. I must say I’m quite pleased with the changes introduced in the new release and would like to review those I consider particularly useful.
July 8, 2015 | Software Consultancy
Most of the important players in this space are large IT corporations like Oracle and IBM with their commercial (read expensive) offerings.
While most of CEP products offer some great features, it’s license model and close code policy doesn’t allow developers to play with them on pet projects, which would drive adoption and usage of CEP in every day programming.
Serverless functions are easy to install and upload, but we can’t ignore the basics. This article looks at different strategies related to testing serverless functions.
November 4, 2021 | Kubernetes
We always read that ‘security is everyone’s responsibility’. For any organisation, big or small, security should always be the primary concern—not a mere afterthought. In terms of Kubernetes, securing a cluster is challenging because it has so many moving parts and, apart from securing our Kubernetes environment, we also want to control what an end-user can do in our cluster.
To achieve these goals, we can start with the built-in features provided by Kubernetes like Role-Based Access Control (RBAC), Network Policies, Secrets Management, and Pod Security Policies (PSP). But we know these features are not enough. For example, we may want specific policies like ‘all pods must have specific labels’. And even if we have the policies in place, the next big question is how to enforce them on our Kubernetes cluster in an easy and repeatable manner.
In this blog post, we’ll address this challenge and other questions pertaining to OPA and how it can integrate into Kubernetes.
Message and event-driven systems provide an array of benefits to organisations of all shapes and sizes. At their core, they help decouple producers and consumers so that each can work at their own pace without having to wait for the other – asynchronous processing at its best.
In fact, such systems enable a whole range of messaging patterns, offering varying levels of guarantees surrounding the processing and consumption options for clients. Take for example the publish/subscribe pattern, which enables one message to be broadcast and consumed by multiple consumers; or the competing consumer pattern, which enables a message to be processed once but with multiple concurrent consumers vying for the honour—essentially providing a way to distribute the load. The manner in which these patterns are actually realised however, depends a lot on the technology used, as each has its own approach and unique tradeoffs.
In this article we will explore how this all applies to RabbitMQ and Apache Kafka, and how these two technologies differ, specifically from a message consumer’s perspective.
Our recent client was a Fintech who had ambitions to build a Machine Learning platform for real-time decision making. The client had significant Kubernetes proficiency, ran on the cloud, and had a strong preference for using free, open-source software over cloud-native offerings that come with lock-in. Several components were spiked with success (feature preparation with Apache Beam and Seldon for model serving performed particularly strongly). Kubeflow was one of the next technologies on our list of spikes, showing significant promise at the research stage and seemingly a good match for our client’s priorities and skills.
That platform slipped down the client’s priority list before completing the research for Kubeflow, so I wanted to see how that project might have turned out. Would Kubeflow have made the cut?
Multi-cloud is rapidly becoming the cloud strategy of choice for enterprises looking to modernise their applications.
And the reason is simple – it gives them much more flexibility to host their workloads and data where it suits them best.
In this post, we focus on Google’s application modernisation solution Google Anthos and the role it can play in your cloud transformation strategy.
“WebAssembly is a safe, portable, low-level code format designed for efficient execution and compact representation.” – W3C
In this blog, I’ll cover the different applications of Wasm and WASI, some of the projects that are making headway, and the implications for modern architectures and distributed systems.
While working with a client recently, we experienced some issues when attempting to make use of NLB external load balancer services when using AWS EKS. I wanted to investigate whether these issues had been fixed in the upstream GitHub Kubernetes repository, or if I could fix it myself, contributing back to the community in the process.
With the upcoming Cassandra 4.0 release, there is a lot to look forward to. Most excitingly, and following a refreshing realignment of the Open Source community around Cassandra, the next release promises to focus on fundamentals: stability, repair, observability, performance and scaling.
We must set this against the fact that Cassandra ranks pretty highly in the Stack Overflow most dreaded databases list and the reality that Cassandra is expensive to configure, operate and maintain. Finding people who have the prerequisite skills to do so is challenging.
April 2, 2020 | Machine Learning
Recent years have seen many companies consolidate all their data into a data lake/warehouse of some sort. Once it’s all consolidated, what next?
Many companies consolidate data with a field of dreams mindset – “build it and they will come”, however a comprehensive data strategy is needed if the ultimate goals of an organisation are to be realised: monetisation through Machine Learning and AI is an oft-cited goal. Unfortunately, before one rushes into the enticing world of machine learning, one should lay more mundane foundations. Indeed, in data science, estimates vary between 50% to 80% of the time taken is devoted to so-called data-wrangling. Further, Google estimates ML projects produce 5% ML code and 95% “glue code”. If this is the reality we face, what foundations are required before one can dive headlong into ML?
March 20, 2020
Traditionally, Usability and Security have been set in opposition to each other: with tight security, we end up with painful user experience. In this blog, Guy focuses on financial services as an exemplar of how we can introduce usability into a vertical with challenging security and compliance requirements.
November 13, 2019 | Software Consultancy
Pioneering and pushing technology boundaries – pretty much a given nowadays for the software-driven startup. Here are some insights we’ve observed working with a number of venture capital (VC) companies who have managed to navigate the choppy waters and successfully grow their business including winning further investment along the way.
With our deep hands-on technical expertise and pragmatic focus, OpenCredo has become a natural software acceleration partner for VC funded organisations who are looking to deliver tangible value as effectively as possible. We’ve been brought in to work alongside these innovators at various stages of their journey. As such we’ve gained an appreciation for and acquired, first-hand insight into some of the pressures and challenges faced. From getting and securing that next round of funding, to grappling with the technical decisions and challenges inherent in sensibly evolving offerings to accommodate future growth and scaling.
This blog is written exclusively by the OpenCredo team. We do not accept external contributions.
Creating and managing a Public Key Infrastructure (PKI) could be a very straightforward task if you use appropriate tools. In this blog post, I’ll cover the steps to easily set up a PKI with Vault from HashiCorp, and use it to secure a Kafka Cluster.
While Prometheus has fast become the standard for monitoring in the cloud, making Prometheus highly available can be tricky. This blog post will walk you through how to do this using the open source tool Thanos.
July 31, 2018 | Machine Learning
Machine Learning, alongside a mature Data Science, will help to bring IT and business closer together. By leveraging data for actionable insights, IT will increasingly drive business value. Agile and DevOps practices enable the continuous delivery of business value through productionised machine learning models and software delivery.
May 16, 2018 | Microservices
To identify service boundaries, it is not enough to consider (business) domains only. Other forces like organisational communication structures, and – very important – time, strongly suggest that we should include several other criteria in our considerations.
February 6, 2018 | Cloud
Among the many announcements made at Re:Invent 2017 was the release of AWS Privatelink for Customer and Partner services. We believe that the opportunity signalled by this modest announcement may have an impact far broader than first impressions suggest.
January 11, 2018 | Data Engineering
The last few years have seen Python emerge as a lingua franca for data scientists. Alongside Python we have also witnessed the rise of Jupyter Notebooks, which are now considered a de facto data science productivity tool, especially in the Python community. Jupyter Notebooks started as a university side-project known as iPython in circa 2001 at UC Berkeley.
The recent 0.10.0 release of HashiCorp Terraform, saw a significant change to the way Providers are managed. Specifically, the single open source code repository for Terraform has been divided into core and multiple provider repositories.
June 15, 2017 | Data Engineering
CockroachDB is a distributed SQL (“NewSQL”) database developed by Cockroach Labs and has recently reached a major milestone: the first production-ready, 1.0 release. We at OpenCredo have been following the progress of CockroachDB for a while, and we think it’s a technology of great potential to become the go-to solution for a having a general-purpose database in the cloud.
Apache Spark is a powerful open source processing engine which is fast becoming our technology of choice for data analytic projects here at OpenCredo. For many years now we have been helping our clients to practically implement and take advantage of various big data technologies including the like of Apache Cassandra amongst others.
In recent years, Cassandra has become one of the most widely used NoSQL databases: many of our clients use Cassandra for a variety of different purposes. This is no accident as it is a great datastore with nice scalability and performance characteristics.
However, adopting Cassandra as a single, one size fits all database has several downsides. The partitioned/distributed data storage model makes it difficult (and often very inefficient) to do certain types of queries or data analytics that are much more straightforward in a relational database.
March 20, 2017 | DevOps
DevOps has swept the tech landscape. Now, many are discovering the benefits of programmable infrastructure. I have been lucky to work on many projects where we’ve taken advantage of tools such as Terraform, Ansible, or Chef.
February 13, 2017 | Data Engineering
One of the stated intentions behind the design of Java 8’s Streams API was to take better advantage of the multi-core processing power of modern computers. Operations that could be performed on a single, linear stream of values could also be run in parallel by splitting that stream into multiple sub-streams, and combining the results from processing each sub-stream as they became available.
January 24, 2017 | Cloud
This blog aims to provide an end to end example of how you can automatically request, generate and install a free HTTPS/TLS/SSL certificate from Let’s Encrypt using Terraform. Let’s Encrypt is a free, automated, and open certificate authority (CA) aiming to make it super easy (and free – did I say free!) for people to obtain HTTPS (SSL/TLS) certificates for their websites and infrastructure. Under the hood, Let’s Encrypt implements and leverages an emerging protocol called ACME to make all this magic happen, and it is this ACME protocol that powers the Terraform provider we will be using. For more information on how Let’s Encrypt and the ACME protocol actually work, please see how Let’s Encrypt works.
October 13, 2016 | Data Analysis
In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program. As you’re writing a program you may think “I wish Lisp had such-and-such an operator.” So you go and write it. Afterward you realize that using the new operator would simplify the design of another part of the program, and so on. Language and program evolve together…In the end your program will look as if the language had been designed for it. And when language and program fit one another well, you end up with code which is clear, small, and efficient – Paul Graham, Programming Bottom-Up
August 26, 2016 | Kubernetes
This post is the last of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible. To understand the goal of the project, you’d better start from the first part.
August 26, 2016 | Kubernetes
This post is the second of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible. To understand the goal of the project, you’d better start from the first part.
August 26, 2016 | Kubernetes
This post is the first of a series of three tutorial articles introducing a sample, tutorial project, demonstrating how to provision Kubernetes on AWS from scratch, using Terraform and Ansible.
July 3, 2016 | DevOps
Several of us from the OpenCredo team were in attendance at the inaugural EU edition of the DevOps Enterprise Summit conference. We have been big fans of the two previous US versions, and have watched the video recordings of talks (2014, 2015) with keen interest as many of our DevOps transformation clients are very much operating in the ‘enterprise’ space.
June 3, 2016 | Software Consultancy
In this post, I’m going to take something extremely simple, unfold it into something disconcertingly complex, and then fold it back into something relatively simple again. The exercise isn’t entirely empty: in the process, we’ll derive a more powerful (because more generic) version of the extremely simple thing we started with. I’m describing the overall shape of the journey now, because programmers who don’t love complexity for its own sake often find the initial “unfolding” stage objectionable, and then have trouble regarding the eventual increase in fanciness as worth the struggle.
April 5, 2016 | Software Consultancy
This post is part of a series which introduce key concepts in successful test automation. Each post contains sample code using the test-automation-quickstart project, a sample Java test automation framework available from Github.
March 3, 2016 | Software Consultancy
JetBrains (the people behind IntelliJ IDEA) have recently announced the first RC for version 1.0 of Kotlin, a new programming language for the JVM. I say ‘new’, but Kotlin has been in the making for a few years now, and has been used by JetBrains to develop several of their products, including Intellij IDEA. The company open-sourced Kotlin in 2011, and have worked with the community since then to make the language what it is today.
Many of our clients are currently implementing applications using a ‘microservice’-based architecture. Increasingly we are hearing from organisations that are part way through a migration to microservices, and they want our help with validating and improving their current solution. These ‘microservices checkup’ projects have revealed some interesting patterns, and because we have experience of working in a wide-range of industries (and also have ‘fresh eyes’ when looking at a project), we are often able to work alongside teams to make significant improvements and create a strategic roadmap for future improvements.
January 26, 2016 | Data Engineering
In this second post about Hazelcast and Spring, I’m integrating Hazelcast and Spring-managed transaction for a specific use case: A transactional Queue. More specifically, I want to make the message polling, of my sample chat application, transactional.
January 18, 2016 | Software Consultancy
Last time in this series I summarised all the Akka Persistence related improvements in Akka 2.4. Since then Akka 2.4.1 has been released with some additional bug fixes and improvements so perhaps now is a perfect time to pick up this mini-series and introduce some other new features included in Akka 2.4.x.
January 8, 2016 | Microservices
Many of our clients are in the process of investigating or implementing ‘microservices’, and a popular question we often get asked is “what’s the most common mistake you see when moving towards a microservice architecture?”. We’ve seen plenty of good things with this architectural pattern, but we have also seen a few recurring issues and anti-patterns, which I’m keen to share here.
November 3, 2015 | Software Consultancy
My JavaOne experience was rather busy this year, what with three talks presented in a single day! The first of these talks “Debugging Java Apps in Containers: No Heavy Welding Gear Required” was delivered with my regular co-presenter Steve Poole, from IBM, and we shared our combined experiences of working with Java and Docker over the past year.
October 31, 2015 | Microservices
Over the past few weeks I’ve been writing an OpenCredo blog series on the topic of “Building a Microservice Development Ecosystem”, but my JavaOne talk of the same title crept up on me before I managed to finish the remaining posts. I’m still planning to finish the full blog series, but in the meantime I thought it would be beneficial to share the video and slides associated with the talk, alongside some of my related thinking. I’ve been fortunate to work on several interesting microservice projects at OpenCredo, and we’re always keen to share our knowledge or offer advice, and so please do get in touch if we can help you or your organisation.
October 28, 2015 | Software Consultancy
Let’s have a quick look at the most interesting changes and new features that are now available to Akka users. As there are many new features to highlight in the new Akka release I will focus on those related to Akka Persistence first and cover other areas in a separate post.
October 12, 2015 | DevOps
DevOps is transformative. This (hopefully) won’t be true forever, but it is for now. While the modern management practices of separating development and operations (and to a lesser extent, everyone else) prevail, the tearing down of the walls that separate them will remain transformative. In company after company, management and front-line staff are coming to realise that keeping functions separate, which are inherently interdependent, is a model for blame, shifted responsibility, and acrimony. It’s easy to divvy-up a company up based on function. To many people, it seems the most logical way to do it. Ops does operations, Dev does development, Marketing markets, etc. It seems much harder to do it any other way. So why do it?
September 24, 2015 | Microservices
Unless you’ve been living under a (COBOL-based) rock for the last few years, you will have no doubt heard of the emerging trend of microservices. This approach to developing ‘loosely coupled service-oriented architecture with bounded contexts’ has captured the hearts and minds of many developers. The promise of easier enforcement of good architectural and design principles, such as encapsulation and interface segregation, combined with the availability to experiment with different languages and platforms for each service, is a (developer) match made in heaven.
September 20, 2015 | Microservices
Over the past five years I have worked within several projects that used a ‘microservice’-based architecture, and one constant issue I have encountered is the absence of standardised patterns for local development and ‘off the shelf’ development tooling that support this. When working with monoliths we have become quite adept at streamlining the development, build, test and deploy cycles. Development tooling to help with these processes is also readily available (and often integrated with our IDEs). For example, many platforms provide ‘hot reloading’ for viewing the effects of code changes in near-real time, automated execution of tests, regular local feedback from continuous integration servers, and tooling to enable the creation of a local environment that mimics the production stack.
If you are operating in the programmable infrastructure space, you will hopefully have come across Terraform, a tool from HashiCorp which is primarily used to manage infrastructure resources such as virtual machines, DNS names and firewall settings across a number of public and private providers (AWS, GCP, Azure, …).
August 11, 2015 | DevOps
For years, OpenCredo has been working with organisations to help them introduce new technologies, and more effective development practices, to their IT teams. This has met with a great deal of success, and we have worked with a variety of companies of various sizes. During these projects, we have consistently noticed that the changes we make reach beyond IT in their impact and effects.
As a company, we at OpenCredo are heavily involved in automation and devOps based work, with a keen focus on making this a seamless experience, especially in cloud based environments. We are currently working within HMRC, a UK government department to help make this a reality as part of a broader cloud broker ecosystem project. In this blog post, I look to provide some initial insight into some of the tools and techniques employed to achieve this for one particular use case namely:
With pretty much zero human intervention, bar initiating a process and providing some inputs, a development team from any location, should be able to run “something”, which, in the end, results in an isolated, secure set of fully configured VM’s being provisioned within a cloud provider (or providers) of choice.
A few weeks ago, we thought about building a Google analytics dashboard to give us easy access to certain elements of our Google Analytics web traffic. We saw some custom dashboards for bloggers, but nothing quite right for our goal, since we wanted the data on a big screen for everyone in the office to view.
July 13, 2015 | Software Consultancy
Why Use Dynamic Proxies?
Dynamic proxies have been a feature of Java since version 1.3. They were widely used in J2EE for remoting. Given an abstract interface, and a concrete implementation of that interface, a call to some method on the interface can be made “remote” (i.e. cross-JVM) by creating two additional classes. The first, a “marshalling” implementation of the interface, captures the details of the call in the source JVM and serializes them over the network. The second, an “unmarshalling” endpoint, receives the serialized call details and dispatches the call to an instance of the concrete class on the target JVM.
November 19, 2014 | Microservices
Undeniably, there is a growing interest in microservices as we see more organisations, big and small, evaluating and implementing this emerging approach. Despite its apparent novelty, most concepts and principles underpinning microservices are not exactly new – they are simply proven and commonsense software development practices that now need to be applied holistically and at a wider scale, rather than at the scale of a single program or machine. These principles include separation of concerns, loose coupling, scalability and fault-tolerance.
Last year some of us attended the London Spring eXchange where we encountered a new and interesting tool that Pivotal was working on: Spring Boot. Since then we had the opportunity to see what it’s capable of in a live project and we were deeply impressed.
November 14, 2013 | Cassandra
Cassandra 2.0 was released in early September this year and came with some interesting new features, including “lightweight transactions” and triggers.
Despite the rising interest in the various non-relational databases in recent years, there are still numerous use-cases for which a relational database system is a better choice. The latest major release of Cassandra (version 2.0) provides some interesting features that aim to close this gap, and offers its fast and distributed storage engine enhanced with new options that will make users’ lives easier.
September 19, 2013 | Software Consultancy
This post will give an overview of mobile testing using Appium. We will integrate tests for a native Android application into an existing Cucumber-JVM based set of acceptance tests and demonstrate multi platform testing from a single set of BDD scenarios. The sample code for this can be found here.
July 2, 2013 | Software Consultancy
If your team is using continuous integration this becomes especially noticeable, forcing teams to either wait for acceptance tests to complete before deploying or having to ignore the bulk of the tests.
The sample code and description in this post will show you how to convert your suite to running tests in parallel, something which has historically been problematic with unanswered questions and outstanding Cucumber-JVM bugs on the subject. Tying the described approach in with Selenium Grid2 will allow you distribute your testing across several machines if your suite is especially large or slow.
June 30, 2013 | Software Consultancy
Spring Data Hadoop (SDH) is a Spring offshoot project that allows the invocation and configuration of Hadoop tasks within a Spring application context. It offers support for Hadoop jobs, HBase, Pig, Hive, Cascading and additionally JSR-223 scripting for job preparation and tidy-up.
It is most suited for use in organisations with existing Spring applications or investment in Spring expertise. Some SDH features replicate functionality of tools in the Hadoop ecosystem that DevOps engineers who maintain a Hadoop cluster will be more familiar with.
February 25, 2013 | Neo4j
As part of our work, we often help our customers choose the right datastore for a project. There are usually a number of considerations involved in that process, such as performance, scalability, the expected size of the data set, and the suitability of the data model to the problem at hand.
This blog post is about my experience with graph database technologies, specifically Neo4j. I would like to share some thoughts on when Neo4j is a good fit but also what challenges Neo4j faces now and in the near future.
December 18, 2012 | Software Consultancy
The first thing most people think of when they start a project with the good intentions of test driven development is: write a test first. That’s great, and something I would fully encourage. However, diving in to writing tests without forethought, especially on large projects with a lot of developers can lead to new problems that TDD is not going to solve. With some upfront thinking (but not big upfront design!) a large team can avoid problems later down the line by considering some important and desirable traits of a large and rapidly changing test suite.
November 4, 2011 | Neo4j
In the previous post we compared the performance of fetching relationships from densely populated nodes using Neo4j native store and using lucene index.
We’ve seen that we can fetch the small subset of relationships from a super-node (containing ~1M relationships in total) directly from the Lucene index, the performance of the first run (cold-caches) is better then using the Neo store directly. The subsequent runs with caches warmed up show comparable performance, slightly in favor of direct Neo store fetching, sue to low level cache optimizations.
June 3, 2011 | Neo4j
Neo4J is one of the first graph databases to appear on the global market. Being open source, in addition to its power and simplicity in supporting graph data model it represents good choice for production-ready graph database.
However, there has been one area I have struggled to get good-enough performance from Neo4j recently – super nodes.