John Järpling, Director of Products

Building a headless e-commerce solution – how hard can it be?

We are reaching the end of a long development journey, with the goal to create our headless and composable e-commerce framework Storefront Excite. In this article I'll share some insights from the journey we have been on, in case you are thinking about doing the same thing yourself.

The creation of Storefront Excite.

Setting out to build a complete headless e-commerce solution turned out to be a not-so-smooth and not-so-simple journey. I don’t recommend anyone to try to pull this off by themselves. That is, if you are not staffed with some of the most experienced and brightest senior architects and developers, specialized in modern commerce solutions.

Fortunately, here at Avensia we have a whole bunch of those. We have learned a lot during decades of e-commerce development. Throughout the years, we have gone through numerous iterations of technical frameworks for e-commerce, and we have built e-com sites for so many customers that I've lost count.

All the time, we have pushed the boundaries for what is possible, because that is part of the Avensia development culture.

This is a story about just that. How we built Storefront Excite using our understanding of e-commerce both from a business and technical perspective, and created something completely new, that no-one has done before. With a win-win-win mindset. Win for Avensia, win for our customers, and most importantly - win for their customers.

This text is a long one, but if you're only interested in parts of it, pick and choose here;

excite--john-hero
Chapter 1

Why did we build Storefront Excite?

A couple of years ago we decided that it was time to take on the task of creating the next generation e-commerce solution, using a headless and composable architecture. Our customers love the way our existing framework Storefront Nitro is built; a complete, but customizable, e-com solution that speeds up time to market.

We wanted Excite to work the same way, as an accelerator that can kickstart new projects, with a lot of pre-packaged functionality that are simple to work with and easy to upgrade. A pre-packaged solution that is shared among several customers works as a solid foundation that we can use to build unique and relevant features on, and thereby adding value to all customers who are using it.

Chapter 2

What components should we use in our composable commerce solution?

We wanted to build Excite on top of the new generation of headless systems, and we wanted the solution to be as focused on modern commerce as we are. After long and thoughtful considerations we decided that commercetools was the right platform for us.

We already had very good experience with Apptus eSales for search and relevance, so we quickly decided to lower the complexity for ourselves by using that platform. Not only because we know it, but because we think it is the best product out there for search and relevance. We use Apptus eSales today in our Storefront Nitro product built with Episerver and it performs exceptionally well in terms of increased conversion, speed and uptime.

What CMS to go with was harder question. We talked to a lot of different headless CMS providers. We weighed technical features against cost and market penetration etc, and we discussed with the customers that we had onboard for this journey. We landed in a decision that now feels like the obvious choice: Contentful. Before we started the implementation and the testing, I can admit we had some doubts, but as it turns out this was a really good choice both from a technical perspective and in terms of the superb support it provides editors.

excite-computer
Chapter 3

Building Excite in close collaboration with customers

Yes, our customers were highly involved in the project to build the Storefront Excite product. It was, and still is, a close collaboration with our customers. Because, if we don’t build what they want there is no point in doing it. There can of course be a difference between what customers think they want and what they actually need. So first we had to agree on the ambition.

Our customers want the best e-commerce site in the world. Then of course, we want to build a product that can give them that. But first we needed to define what we both mean by “the best”. It’s easy to get blinded by pretty design, or a Google speed score, but just as important are the processes behind. How to process returns, launch campaigns etc. Is there an OMS that supports the business?

To put it simply, we at Avensia consider a superior e-commerce solution to have:

  • the fastest technical performance
  • the best customer experiences
  • the best support for customer service
Chapter 4

Building something customers’ dev teams can easily work with

Another aspect of the close collaboration with our customers for Storefront Excite is that we believe that most customers that are interested in the headless world are more digitally mature and will have an internal development team in place. This means that the way we choose to build Storefront Excite will have to appeal to our customers’ internal development departments.

We also need to set up the code base and the projects in a way that supports collaboration between our project team and the customers’ developers.

Chapter 5

Building one product and two customer projects at the same time

We started with a small development team of 3 senior solution architects that began drawing up the basics, fired up Visual Studio and started coding. As it turned out, we found an initial customer who wanted help with just a few backend things for commercetools. This gave us an opportunity to learn about payment provider integrations and order integrations in commercetools.

In parallel, we started working with the next customer who wanted a full solution (frontend and backend). During the autumn they decided they needed to launch a CMS site for a collaboration with a brand they had. First, they thought of just doing a WordPress quickie. But in the future, they wanted to have an option to add e-commerce functionality to that site, so after some discussions we decided that we should do this with Storefront Excite architecture, but only the CMS part, and try out basic Contentful capabilities.

Both these somewhat small side quests, although they were stressful, gave us good opportunities to learn and improve. In the middle of November 2020, the development team had doubled and the technical challenges were starting to mount up. We had delivered backend to one client that had gone live and a CMS site for another. Now the sales team came with yet another client. This retailer needed a complete solution. We were no way near a complete product and still had lots of problems to solve…

To be as efficient as possible, a decision was made to merge the product development and the project teams for the 2 clients into one big team of developers. We dedicated one solution architect to each customer and one for the product. We also decided to put all other developers in one common resource pool. The reasoning behind this setup was that we could save lots of time if the same developer did the product development of for example Login to My Pages and then implemented that for customer 1 and 2 in parallel.

We gathered all Project Managers and the Solution Architects and Client Directors into one common forum to discuss and work out resource and feature issues together. Using an agile mindset we put no one in charge but insisted that the forum needed to talk it through. So, everyone is responsible for everyone’s success. So far it has worked out pretty well. It is not easy, but we have experienced many of the expected synergies, and been able to keep product development on track as well as happy customers. All this in a completely distributed team working from home and only coordinating via Slack and Teams meetings.

Chapter 6

Examples of technical challenges when building a headless solution.

At times it felt you could pick a rock and throw it at the Jira board and anything you hit was a problem (and now a broken monitor on top of that 😊 ). Jokes aside, let’s take a very good example of problems you face when joining several headless systems: URLs and routing.

In a sense commercetools and Contentful are two databases containing items. commercetools contains for example Products (t-shirts/bikes/coffee cups or whatnot) and Contentful Pages (Category Page, Campaign Page, Start Page, Coffee cup Brand X Page etc). The items need to form a part of the URL so you can navigate to that item, this is in technical terms called the slug. www.example.com/category-x or www.example.com/category-x/coffe-cup the red part here being the slug. Neither commercetools nor Contentful will solve the actual routing for you, you need to build this yourself.

In a simplified scenario you would build a routing component that worked like this: when the browser asks for a specific URL you parse out the slug and send that request on to commercetools and then Contentful and sort out if any of them has that item and then you fetch the item from the correct source and show it. Easy peasy right?

There are actually several problems layered here;

  • Let’s say Category X suddenly is named Category Y (a very common scenario). You would suddenly get a 404 page. Ask your SEO managers what they think of that. Bring earplugs.
  • Lookup of a slug in commercetools and Contentful is fast, but still too slow (10 ms – 200 ms is what we have found). This is a very long time to wait, every millisecond counts. We have it down to average 3 ms now in our solution, and we are proud of that.
  • Sometimes you don’t just need one slug entry, if you are using a package that contains several other items then you need to fetch each of them to.

We have created a very advanced routing component that actually builds and tracks all URLs in a separate cloud database. We construct objects for packages and pre-index all URLs, change tracking with automatic redirects for when a product changes category etc. We then cache all URLs to all slugs in commercetools and Contentful to enable even better performance.

This also gives us the possibility to sync changes done in commercetools into Contentful. These are separate systems that are not aware of each other, so when creating a category in commercetools an editor in Contentful has no idea that it exists or how to use it. With our solution we are automatically creating categories and updating them in Contentful so that there is always an updated category tree to use for editors when creating content.

This is just one of many things we had to solve. And we have also made sure that the solution ensures performance, allows scaling and thereby really helps the customer in their daily life.

The development is now coming to an end. The security, performance, UAT etc testing is ongoing and go-live for the first customers is very close. And we are onboarding new customers into the team.

Excite@2x-80-grey
Chapter 7

What can development teams and IT departments learn about headless solutions, based on our experiences?

It's complicated, its damn hard, it takes longer than you think, and you learn a lot.

I like to say that you should strive to make new mistakes, not old ones. And we sure followed that one, made lots of first-time mistakes. The costly ones, the bad ones, the ones that forced us to rebuild some things (several times!). But we are past that now. We now have an architecture that performs really well, a solid base to build on, securing that we can create the most outstanding user experiences and support customer services.

This has been a long project, but I do believe we have created something fantastic. The Storefront Excite solution is a best-of-breed product, built with composable commerce as guiding principle, and I really do believe it will set a new standard for headless commerce going forward. You cannot beat this product.

Until we at Avensia builds the next product perhaps 😉