Here is an evaluation checklist to help you ensure you make the right decision if you are changing development technologies, adding a new tool to your stack, or starting from scratch.
We read about new technologies every day, from tools, to libraries, to complete development stacks. A technology may grab our interest, but a real evaluation is needed before we consider adoption.
I’ve consulted with a number of organizations on technology stack evaluations and eventual migrations. The number 1 regret every organization has is not spending the time to truly evaluate and vet the technology stack they chose. Taking the time to answer some questions up front would have helped identify future limitations.
That’s the purpose of this checklist. By following the checklist, you can make an informed decision about your technology choices. In this article, I’ll show you how to use my checklist to evaluate a technology and even compare competing technologies.
Introduction
The technology adoption checklist is divided into 8 categories. Each category represents an aspect of the evaluation that will be performed. You can find a quick summary of each category below. If you would like a printable PDF checklist, you can download that here:
Tech Stack Evaluation Checklist Download
Once we complete the evaluation of a category, we give it a rating. This rating should simply make sense to you and your team. It’s your way of determining whether a category meets your organizational criteria.
The rating can be a pass / fail, a number from 1-10, a number of stars, really anything you choose. The key is making a determination of what’s acceptable for your organization and defining your criteria around that.
For example, scalability may not be a concern if an application is being developed for internal use only and we have minimal performance requirements. You should still look at performance and scalability metrics to ensure you understand the ramifications, but you may simply have a lower passing bar / weight for the category.
If you are comparing multiple competing technologies, your ratings provide you a measuring stick. If you are simply deciding whether to bring a new technology into your existing stack, your ratings can help you decide whether it’s worth it.
Now let’s look at the categories and the questions we should be asking.
1. Industry Adoption
Identify how widely used the technology is within the industry. Adopting a niche technology introduces risk, especially if your development team will be using it for years to come. A lack of industry adoption impacts every other category on this list.
For example, limited adoption may cause the original developers to lose interest and abandon the project. This means your team may end up having to work around or even fix bugs when they are found.
Key questions to answer:
- Who developed the technology and are they still using it?
- What Fortune 500 companies have adopted the technology?
- Do you have access to weekly / monthly download data (e.g. npmjs.org)?
- How many job posts can you find referencing the technology on popular job boards (e.g. indeed.com)?
- What is the adoption trend? Is it continuing to increase, stagnating, or decreasing?
- Is the technology being featured at conferences?
2. Licensing Terms
Licensing terms determine how you can use the technology now and in the future as well as cost expectations. Some companies prefer open source as it feels less risky should the backing company disappear.
Others prefer proprietary technologies as they ensure support when an issue comes up. Open source, paid support models offer something in-between. Your team has to decide what is important to you.
Key questions to answer:
- Is the product licensed through an open source, open source with paid support, freemium, or proprietary model?
- What are the costs associated with using the software? What do we get for what we’re paying?
- If proprietary, who is the backing company and how stable are they? What is their market share?
- If open source, what are the terms? In what ways will the licensing impact our use of the software?
3. Documentation
No documentation, no thanks. Detailed and well-written documentation is a must. A lack of documentation will lead to slower development at best and an eventual need to abandon the technology at worst. The most misunderstood line of the Agile Manifesto is:
Working software over comprehensive documentation.
Agile Manifesto
This line only states that working software is valued over documentation. In no way does it mean that documentation isn’t necessary. In fact, it’s critical to the usability of most development technologies.
Key questions to answer:
- Does the technology include a detailed User Guide?
- Does the User Guide cover important topics (e.g. installation, core functionality, usage examples)?
- Is the documentation well-formatted and written in clear and easy-to-follow language?
- Is it easy to find topics that you’re looking for within the documentation?
- Does the documentation include screenshots and/or code snippets along with the text?
4. Ease of Use
The proof is in the pudding. The documentation may look great, but until you actually build something with a technology it’s impossible to truly gauge the ease of use. A development team should build a proof of concept from the documentation implementing actual features.
This is the only way to get a realistic feel of how the technology can be applied to your specific needs. The ease of use for a particular technology will be completely subjective based on the experiences of the team. For example, a team with extensive JavaScript experience will find it much easier to adopt a technology like NodeJS than a team with only Java experience.
Remember that ease of use will affect your development speed and time to market. Knowing that, it’s important to spend enough time with the technology to get a good feel for it and how it will impact your team.
Key questions to answer:
- Does the technology really help you to solve your problem?
- How much time does is take to get something up-and-running?
- How much adaptation is required to apply the technology to your specific use cases?
- What is the knowledge gap between this technology and what your team has used in the past? This is not necessarily a drawback, just something to be aware of.
- Are there any hurdles you had to address along the way?
5. Community Support
Community support is especially important for open source technologies. Even proprietary technologies benefit from a large developer community, as it’s much more convenient to find a quick answer to your answer online.
Google and StackOverflow are a developer’s best friends. A development community answers questions in online forums and on StackOverflow. They provide example solutions on GitHub and blogs. With open source projects, the community even submits patches, implements new features, and drives the future direction of the product.
Remember, the longer a developer spends trying to find solutions to common use cases or solve errors, the fewer features they’re developing.
Key questions to answer:
- What is the product’s Google and StackOverflow presence?
- Does a search for the technology return numerous, quality results?
- Do developers provide working examples online in blogs or on GitHub?
- Are there forums dedicated to answering questions about the technology?
- For open source projects, is there a large group of developers who contribute to the project?
6. Feature Mapping
You should have a good idea of what you need from the technology. Create a list of the features you expect that you will need now and in the foreseeable future. Your feature needs could be certain capabilities, specific architecture, security requirements, supported programming language… It’s up to you and your team to determine what features you are planning to require from a technology.
Once you have this, identify the supporting features in the technology and clearly map whether it meets the need and how. If a required feature can’t be found, have a look at the Extensibility (see below). It may be possible to find a plug-in, module, or extension that can be added to meet the need.
Key questions to answer:
- What features do you expect the technology to support? Think about capabilities, security, architecture, etc.
- How does the technology support the features you require?
- Are there any gaps and can they be filled through extensibility?
7. Performance and Scalability
The technology we choose must be able to adapt as our system requirements grow and the user base expands. We may be faced with a technology migration to move forward if we end up hitting a performance wall.
We can only possibly foresee what the immediate future holds, and even that’s questionable. But, it’s best to err on the side of caution here. As a rule of thumb, I tend to 10x what we think the performance needs may be in the future for evaluation purposes.
Key questions to answer:
- What is the size of your expected user base and / or the context of your usage?
- Have any performance metrics been captured by others about the technology?
- What companies are using the technology and what is their usage context?
- Can you develop stress tests to confirm your performance requirements?
8. Extensibility
Last, but certainly not least, is extensibility. When choosing tools or a technology stack it’s especially important to gauge just how easy it is to extend the technology to suit your needs. Extension may be achievable through plug-ins, modules, libraries, or configuration.
Community support is also a factor in the extensibility of a technology. A strong developer community will tend to create publicly available libraries that provide integration with other technologies and simplify development tasks.
Key questions to answer:
- How can the technology be extended? Does it have a plug-in, module, or library system?
- Is the development community actively creating extensions for the technology?
- How easy is it to build your own extensions if needed?
Conclusion
It’s impossible to know what the future will bring, but that doesn’t mean we shouldn’t plan for it. Even if we have to make some assumptions, simply taking the time to evaluate a technology choice before adoption can avoid a lot issues down the road. By following this guide, you can make more informed choices and avoid a migration down the road.
Learn more:
- Choosing a Tech Stack – Evaluation Checklist Download
- Driving new Technology Adoption within your Team
- 6 Symptoms your Technology Stack is Holding you Back
If you need help with a technology evaluation, contact us. For more useful content, you can subscribe to our newsletter below.