Solution Exploration + Evaluation Tables

Introduction

After the root cause analysis is done and a to-be concept model begins to materialize, you may start thinking ahead about which software or platform approach you could take.  It's important that you don't jump too quickly to a single solution, as this could cause you to disregard better, cheaper, or even simpler options.  For example, you may be knowledgable in Salesforce after a recent internship and think, "We could totally use Salesforce to do this!", but what if a CRM isn't the right approach?  You may think that you'll just use the technology taught in MIS 333K since you know best but what if your client can't maintain that.  Again, it's important to not jump to a platform decision too soon but rather consider all avenues since each could have advantages and disadvantages over each other when it comes to aspects like functionality, cost, maintainability, accessibility, security, and more.

The other thing you have to assess, in cases where you have too much scope, is what mix of scope your team will tackle.  For example: If your time is too limited to complete a full web app with every ideal feature, do you focus more on a back-end solution with minimal front-end or do you focus more on complete UI and an MVP back-end?  It will depend on the pros and cons as well as your client's input.

Regardless of the situation, the key is due diligence and thoroughness.  You're expected to explore as many options as you can and with your client, narrow down to the right mix of scope and the top 2 or 3 platform solutions.  After you've narrowed down, a deeper review of the top options will be put into an evaluation table that lists the key features of each option along with advantages, disadvantages, budget (i.e. cost impacts), and questions/unknowns.  This evaluation table is part of the charter and is used to make/document the final platform decision you make with the client.


Before you jump to a specific platform solution...

...you are expected to explore a number of options.  But, how do you start the process of exploring solutions?  There are many aspects to consider but to get started, begin by considering options of Packaged vs Custom.  NOTE: This is a spectrum so it's more than just finding one packaged solution and one Custom one.  We expect you to explore numerous packaged, numerous custom options, and numerous options in between. 

Platform_spectrum.PNG

1. Packaged Solutions: Package solutions include anything that is pre-built and already existing that doesn't require you to make it from scratch. Typical advantages of packaged solutions are that they're readily available which can save time or allow you to get more done in your limited project time.  They can also offer easier maintainability and a managed upgrade path, meaning the vendor provides new features and fixes as part of the product's support.  Disadvantages of packaged solutions may be things like cost (since pre-built solutions are many times sold by vendors), limited functionality (since you're reliant on the vendor's design which wasn't catered directly to your client's needs), and restricted customizability (which could be important if the client wants to gain a key advantage through to software you're building). These pros/cons can clearly vary by solution and client.  When you're researching solutions online consider the following tips:

  • TIP #1: Research smartly - Start by looking at www.capterra.com/categories Links to an external site..  Here you can find a searchable directory of different software programs along with details and ratings.  Beside this directory, you should do your own googling.  Clearly, you can search online for solutions and find a ton of options but at times it's hard to find the specific, niche software that best fits your client needs.  In these cases, think about the right "keywords" that describe your solution.  For example: if you're building an application for a company that allows managers to provided feedback to employees, you may not want to just google "Feedback system software" since this web search may send you down a path of survey tools or customer feedback plugs for websites which isn't what you want.  Part of googling smart, is knowing what to google so as you google more and learn more keywords, refine your keyword searches so you can get closer to the right solution.  Think of it more like a scavenger hunt and be persistent about exhausting all your search options.   
  • TIP #2: Decide if remote access is needed - Does your client want to have easy remote access to the software or do they only need access while they're at their office?  Don't assume the solution needs to be hosted or web-based.  In some cases, a local database would be sufficient for a client and a hosted web application may be overkill.  Also hosting costs money and lots of times this may be a recurring cost clients don't consider.
  • TIP #3: If you are hosting, decide where - You have the option of hosting software yourself on your own hardware on-premise or through a hosting company. You can have your software vendor host it (i.e. SaaS). An advantage of SaaS products is that it's supported with an SLA Links to an external site. as part of a monthly fee so the client doesn't have to have hardware and IT Support staff.  The issue, at times, is that the clients that can't afford IT infrastructure, may not have a budget for a monthly software fee.  Also, remember that SaaS products are packaged so they experience the same limitations as non-SaaS package solutions like limited functionality or reduced customizability.

2. Custom solutions: Custom solutions are anything that allows you to produce a unique solution that doesn't already exist and fits a new or specific need.  It doesn't mean you're writing code from scratch necessarily.  It could mean that you are using a combination of existing code libraries or plugins to produce a unique solution for your client.  The advantages and disadvantages are typically the inverse of packaged solutions.   They allow for unique solutions that fit the client's unique needs, if necessary and they could be cheaper in the long run since you're not purchasing a software license, but they could be harder to maintain and take longer to develop.  Also, keep in mind they are not 100% free since you likely can incur maintenance costs like IT Support Staff or hosting possible.  NOTE: You should really only custom build a solution when you can't buy/rent one due to cost or maintainability constraints. 

  • Custom TIP: Consider your client's skills and needs - As part of your client interaction, you should be assessing if they can manage a custom application or not.  If they don't have an IT support staff or technical skills themselves, consider whether a custom solution is best for them in the long-term.  Also if the client is asking for functionality that requires a custom solution, try to determine what percentage has to be custom.   For example, if you could give the client  80% of their requirements in a packaged, easier-to-maintain solution, would that be considered success?  While the custom solution could provide 100% of the functionality, it may be riskier to get it built in the same time and harder to maintain.  

3. Hybrid solutions: The beauty of a hybrid approach is that you can leverage the advantages of both Packaged and Custom solutions by using readily available code that you can change to fit the client's needs. Today so many existing codesets exist like open source software solutions and open source libraries and packages.  There are also a growing number of "low-code" solutions and content management systems that allow you to leverage builders to create custom solutions without having to start from scratch.  Hybrid solutions exist in many forms and below are just a few common examples:  

  • CMS platforms: Content Management systems like Wordpress, Drupal, Confluence.  See a list of options here Links to an external site.Warning: Make sure you're researching the longevity and quality of the platform you select.  Due to the high amount of rivalry in the market, it's possible a newer or less-proven vendor (while cheaper) could go out of business.  
  • Low-Code Solutions: Knack, OutSystems, Microsoft PowerApps, Appian.  NOTE: While low-code solutions are really awesome at allowing rapid custom development, they typically come with a price.  Tools like Knack are about $500 a year while OutSystems gets much more expensive since it has greater capabilities to support enterprise-level applications.  The same warning applies as with CMS.  Be sure you research the vendor testimonials, listed clientele online, or even call the vendor to ensure it's a proven option.
  • Open Source Code: Some packaged solutions (e.g. SugarCRM), content management systems (e.g. Canvas, Wordpress) and coding frameworks (e.g. Django Python MVC, Angular) are open source so you can download them for free and just pay for hosting.  Keep in mind that while this customizability is an attractive aspect, it requires someone to maintain it.  Also, any major changes to an open source solution could affect your ability to remain on the upgrade path so changes you make should be done with caution.

Concluding Solution Exploration

After you've found an exhaustive list of options map out the options with the client like below.  As you map them out, make sure to identify and notate important considerations like cost and maintainability with the client.  While some solutions provide more functionality, that may come with a monetary cost or a maintainability risk.  Note: the amount of functionality that each solution provides will be dependant on your client's specific needs.  Notice below how some options that differ based cost, functionality, and maintainability.   

Research_Options.PNG

Lastly, after you review with your client you can narrow down to your Top 3 options and move to the next step of creating an evaluation table which will explore these top options in more detail.  


So...what is an Evaluation Table?

System Evaluation tables enable you to quickly and efficiently identify and summarize the advantages and disadvantages of various alternatives during any stage of the system development life cycle. For example,

  • What is the best scope for the project?
  • What is the best platform for implementation?
  • Should we buy application software or create custom software or choose an open source solution?
  • If we purchase application software, what is the best choice to meet project goals?

Purpose

System Evaluation tables facilitate important communication regarding system or scope decisions between the project team and the client. When complex choices must be made regarding the overall solution, create a system evaluation table to identify, summarize, and clarify the alternatives available.  This will enable the project team and the client to enter into purposeful discussions leading to successful decision making.  NOTE: Eval Tables can be used to evaluate any situation where you're considering multiple alternatives whether that be platforms, scope, templates, tools, or general project approaches.


Creating a System Evaluation Table

Make sure that before you create your System Evaluation table, that your team has done a thorough platform analysis which is discussed above. 

After you narrow down to the top three solutions that you plan to present to the client,  use the template linked at the bottom of this page.  As you consider each option, make a note of the advantages and disadvantages of each option.  Also, there will likely be unknowns and questions you encounter with each alternative.  It's very important to note all open questions you have.  This may help indicate which alternative is more feasible or which one will require substantially more research.

Benefits

By identifying and answering critical questions regarding possible system solutions before beginning your sprints, you increase the likelihood that the client will be happy with the chosen end solution.  A discussion of your listed advantages and disadvantages may unearth misconceptions that will save you hours of development time. For example, without an evaluation table comparing actual pros, cons, and costs for a platform, your team might select a solution based on the wrong criteria. The result could be lost time setting up your development environment on one platform and then having to redo your work on another platform that has the capacity and tools necessary for your project.

Tips for Completeness

  • Complete all of the rows for each possible alternative as shown in the example linked below.
  • Make sure you review this table with your client and ensure they understand each option, including advantages, disadvantages, unknowns/questions, and budgetary aspects.
  • For your Project Charter, identify the client’s choice, as illustrated in the example linked below.

High-Quality Delivery Tips

  • Create a System Evaluation table for issues that are not clear decisions—this can be a choice of tools, templates, procedures, the scope for the release, or any system aspect with multiple alternatives.
  • Write a title that clearly identifies the issue under evaluation.
  • Create meaningful alternatives to discuss with your client.
  • Follow the template.
  • Complete each row in the template with information that is relevant for your situation.
  • For the Project Charter, clearly indicate the client’s decision.

Templates and Examples