Solution Analysis Grading Criteria

 

Excellent

Good

Fair

Executive Summary -- This maximum 1-page email to your client sponsor should serve as a very high-level summary of your Solution Analysis work.  At a minimum it should include in a logical order a short description of (1) a quick recap of the original project request, (2) a recap of the root cause(s) that led to the proposed system (3) a short description of the proposed system’s key objectives and constraints, and (4) a summary of major project risks and reduction plans. Include all major items your client should know about your project using appropriate language for them (i.e. use terms they understand). CC your MIS 374 professor and include a link to the Solution Summary Mural.

Clear and concise recap of the project request

Clear recap of project request; too much detail included for an Executive Summary

Vague recap of project request; unclear what the original goal was

Clear and concise recap of the root cause

Clear recap of root cause; too much detail included for an Executive Summary

Vague recap of root cause; unclear what the root cause is an how it relates to the project

Clear description of overall objectives and project constraints

Clear description of overall objectives but missing project constraints

Vague description of overall objectives and project constraints

Clear and concise summary of risks and associated reduction plans

Clear recap of risks; reduction plans not specific

Vague recap of risks and/or missing reduction plans

CC’d prof and included link to Solution Summary Mural.

Cc’d prof but did not include link to Solution Summary Mural

Did not cc prof on email to client

Stakeholders Analysis – Clearly identify all parties impacted by the project. Capture their level of impact on the project as well as their level of interest in or availability for the project. 

Completely aligns with (relevant subset) of org chart (if included)

Misalignment between org stakeholders and people included in org chart (if included)

 

Includes all obvious stakeholders (e.g. all users identified in user stories are included in this analysis)

Missing one or more obvious stakeholders (e.g. does not tie to user stories)

 

Appropriate assessment of the interest/availability of the stakeholder

Clear mistakes in the assessment of the interest/availability of the stakeholder

 

Appropriate assessment of the impact of the stakeholder

Clear mistakes in the assessment of the impact of the stakeholder

 

Model As-Is Client Situation – Create a diagram(s) to model relevant aspects of the current “as-is” situation for your client. Explain why the type of diagram is best. Where do you need clarity the most?  What insights from this model were relevant to your root cause analysis and/or potential solution(s)?

Diagram clearly and thoroughly portrays as-is situation as relevant to the project scope

Diagram portrays as-is situation as relevant to the project scope, but some clarity lacking (either through missing content, illegible items, or leaves reader with open questions)

Diagram missing relevant detail related to focus of the project OR  Model only captures broad aspects of the org rather than only aspects related to the project

Follows all guidelines of model type selected

1-2 guidelines of model type selected not followed

Several guidelines of model type selected not followed OR Model type used is unclear

Provided clear relevance of type of diagram selected

Provided relevance of type of diagram selected, but could be more clearly stated

Vaguely described relevance of type of diagram selected, not very specific to client situation OR

Statement of relevance of diagram type unclear

Clear and convincing reasoning for where clarity is needed

Reasoning for where clarity is needed, but could be more clearly stated

Vaguely described where clarity is needed, not very specific to client situation OR Statement of where clarity is needed is unclear

Excellent reasoning for model insights and how they inform the root cause/potential solutions

Provided general and accurate reasoning for model insights and how they inform the root cause/potential solutions but could be more thoughtful

Vaguely identified insights gained, not very specific to client situation OR Statement of insights gained from model is unclear

Exercise feedback thoroughly considered and thoughtfully integrated

More than one half of exercise feedback integrated into final model

Less than half of exercise feedback integrated into final model

Root Cause Analysis – This section should comprehensively address all issues and concerns for project success down to the basic symptoms driving need.

Symptoms, Problems, Root Cause identification clear, specific, and thorough, problems clearly tie to and explain “why” for symptoms

Symptoms, Problems, Root Cause identification clear, specific, and thorough, but connections between problems and symptoms is unclear (does not explain “why”)

Symptoms, Problems, Root Cause identification vague and/or obvious symptoms/problems seem to be missing, connections between problems and symptoms is unclear (does not explain “why”) OR Confusion between problems/root cause and symptoms (e.g. identified a problem as a symptom), connections between problems and symptoms is unclear (does not explain “why”)

Solution(s) proposed clearly articulated and clearly address the root cause(s) identified

Solution(s) proposed clearly articulated but does not address the root cause(s) identified

Solution(s) proposed vaguely articulated and generally address the root cause(s) identified OR Solution(s) proposed vague and unclear how they address the root cause(s) identified

Each objective clearly ties to a symptom/problem identified, and each symptom/problem has an associated objective

Most objectives clearly tie to a symptom/problem identified, and most symptoms/problems have an associated objective

Objectives are written broadly but still generally tie to symptoms/problems identified OR Objectives are written broadly and do not tie to symptoms / problems identified

All performance criteria (PC) are measurable and tie to an objective; each objective has an associated PC

Most PCs are measurable and tie to an objective; most objectives have an associated PC; some confusion with non-functional requirements

Few PCs are measurable, and few tie to an objective; several objectives missing associated PC; some confusion with non-functional requirements OR PC are not measurable and /or do not tie to objectives; objectives do not have an associated PC; significant confusion with non-functional requirements

Project constraints clearly articulated, specific to the client situation; no obvious constraints missing

Project constraints identified but vaguely articulated, not very specific to the client situation; no obvious constraints missing

Project constraints identified but vaguely articulated, not very specific to the client situation OR Project constraints are unclear or not project constraints OR Missing most obvious constraints

Exercise feedback thoroughly considered and thoughtfully integrated

More than one half of exercise feedback integrated into final root cause

Less than half of exercise feedback integrated into final root cause

To Be Concept Diagram – Concept diagram should illustrate the proposed system. This diagram should tie closely to the objectives of the proposed solution(s) in the root cause analysis and should tie closely to the identified user stories.

Diagram clearly and thoroughly portrays to-be system and all parts of the diagram are legible

Diagram missing 1 or 2 details leading to confusion on what the new system will do OR Parts of diagram are not legible

Diagram missing significant detail of the to-be system OR Diagram is not legible

Diagram clearly ties closely to the objectives of the proposed solution(s) in the root cause analysis

Diagram missing one or more objectives of the proposed solution OR Diagram includes one or more objectives that are not included in root cause analysis

Diagram does not tie to proposed solution objectives at all

Users tie to stakeholders analysis

1 or more users in to-be concept diagram not included in stakeholder analysis

 

All concept modeling guidelines are followed

1 or more concept diagram guidelines not followed, reducing clarity of the model

Model type created is not a concept diagram

Exercise feedback thoroughly considered and thoughtfully integrated

More than one half of exercise feedback integrated into final model

Less than half of exercise feedback integrated into final model

User Stories – Identify all user stories and tie directly to the to-be concept diagram. User Stories should follow correct format.

Correct user story format for all user stories

Up to ¼ of stories do not follow the correct user story format

¼ or more of stories do not follow the correct user story format

All stories tie to the to-be context diagram

Up to ¼ of stories do not tie directly to the to-be context diagram.

¼ or more stories do not tie directly to the to-be context diagram

No stories missing that were identified in the to-be context diagram

Up to ¼ of stories that were identified in the to-be context diagram are missing

¼ or more of stories that were identified in the to-be context diagram are missing

All user stories sized appropriately (e.g. can complete in 1 week; more than one story per user when applicable)

Up to ¼ of user stories sized inappropriately (e.g. cannot complete in a week; only one story per user when should have multiple)

¼ or more of user stories sized inappropriately (e.g. cannot complete in a week; only one story per user when should have multiple)

Exercise feedback thoroughly considered and thoughtfully integrated

More than one half of exercise feedback integrated into final list

Less than half of exercise feedback integrated into final list

User Acceptance Criteria – for each user story should be identified (1-3 per user story). For each story think about how the user would not only use the functionality but how they would likely determine it meets their expectation. Make sure the client contributes to the UACs and agree to what would make the story "acceptable"

UAC relates specifically to the functionality of the story

¾ or more relate specifically to the functionality of the user story

Less than ¾ relate specifically to the functionality of the user story

UAC clearly outlines how user story will meet their expectations

¾ or more clearly outline how the user story will meet their expectations

Less than ¾ clearly outline how the user story will meet their expectations

Exercise feedback thoroughly considered and thoughtfully integrated

More than one half of exercise feedback integrated into final model

Less than half of exercise feedback integrated into final model

Feasibility Matrix – Co-create a feasibility matrix with your client that includes all proposed user stories.

All stories included

¾ or more user stories included in matrix

Less than ¾ of user stories included in matrix

Assessment of difficulty of all user stories seems appropriate for the project

Assessment of difficulty of ¾ or more user stories seems appropriate for the project

Assessment of difficulty less than ¾ of user stories seems appropriate for the project

Assessment of value of all user stories seems to align with other project analysis

Assessment of value of ¾ or more user stories seems to align with other project analysis

Assessment of value of less than ¾ of user stories seems to align with other project analysis

Exercise feedback thoroughly considered and thoughtfully integrated

More than one half of exercise feedback integrated into final matrix

Less than half of exercise feedback integrated into final matrix

Non-Functional Requirements – Address Maintainability, Portability, Scalability, Security, Performance, and Accessibility context and requirements for the project.

Maintainability context – clearly and correctly stated, specific to client situation

Maintainability context – vague, not correct AND/OR not specific to the client situation AND/OR misunderstood maintainability

 

Maintainability requirements – specific to client and is something you could build/configure in the new system or create/do for the client

Maintainability requirements – not specific to client and/or is vague and/or is not a requirement and/or not correct

 

Portability context – clearly and correctly stated, specific to client situation

Portability context – vague, not correct AND/OR not specific to the client situation AND/OR misunderstood portability

 

Portability requirements – specific to client and is something you could build/configure in the new system or create/do for the client

Portability requirements – not specific to client and/or is vague and/or is not a requirement and/or not correct

 

Scalability context – clearly and correctly stated, specific to client situation

Scalability context – vague, not correct AND/OR not specific to the client situation AND/OR misunderstood scalability

 

Scalability requirements – specific to client and is something you could build/configure in the new system or create/do for the client, includes actual #s

Scalability requirements – not specific to client and/or is vague and/or is not a requirement and/or does not specify #s and/or not correct

 

Security context – clearly and correctly stated, specific to client situation

Security context – vague, not correct AND/OR not specific to the client situation AND/OR misunderstood security

 

Security requirements – specific to client and is something you could build/configure in the new system or create/do for the client

Security requirements – not specific to client and/or is vague and/or is not a requirement and/or not correct

 

Performance context – clearly and correctly stated, specific to client situation

Performance context – vague, not correct AND/OR not specific to the client situation AND/OR misunderstood performance

 

Performance requirements – specific to client and is something you could build/configure in the new system or create/do for the client, includes actual #s

Performance requirements – not specific to client and/or is vague and/or is not a requirement and/or does not specify #s and/or not correct

 

Accessibility context – clearly and correctly stated, specific to client situation

Accessibility context – vague, not correct AND/OR not specific to the client situation AND/OR misunderstood accessibility

 

Accessibility requirements – specific to client and is something you could build/configure in the new system or create/do for the client

Accessibility requirements – not specific to client and/or is vague and/or is not a requirement and/or not correct

 

Evaluation Table – Used to help determine system solution. Make research of alternatives clear. Be sure to clearly indicate what alternative the client chose.

Included a list of 10 or more possible options varying by solution type (includes top 3 solutions)

Included a list of less than 10 possible options varying by solution type (includes top 3 solutions)

 

Clearly identified the number of high/medium/low value user stories are covered by each solution Did not clearly identify the number of high/medium/low value user stories are covered by each solution

 

Identified unique advantages of each solution not shared by the other options and tie to client needs

Advantages included are vague and/or redundant across solutions and/or do not tie to client needs

 

Identified unique disadvantages of each solution

Disadvantages included are vague and/or do not tie to client needs

 

Costs – provided upfront (dev) costs and recurring (operational) costs

Costs – missing obvious costs and/or presentation of costs is difficult to understand

 

Costs – values included across solutions use a consistent unit of measurement (e.g. monthly, by user)

Costs – used different units of measurement across solutions (e.g. monthly fees for one and number of users for another)

 

Client decision clearly shown

 

 

Thoughtful questions included and team transparent about unknowns

Questions are vague and/or obvious unknowns not included

 

Client Background/Overview - Address the following items: a) identify the organization structure broadly, b) pinpoint the part(s) of the organization that the proposed system affects, and c) address impact of the system and benefits to areas of strategy, management, and operations of the organization that is particularly relevant in the context of the system.

Clear and concise background on org’s mission and overall structure

Vague background on org’s mission and overall structure OR did not consider overall structure of the org

 

Clear identification of the part(s) of org affected by the proposed system

Vague identification of the part(s) of org affected by the proposed system

 

Strategic impact – clearly considered both positive and negative impact, specific to client situation

Only considered either positive or negative impact, specific to client situation

Vaguely written OR not specific to the client situation OR misunderstood strategic impact

Managerial impact – clearly considered both positive and negative impact, specific to client situation

Only considered either positive or negative impact, specific to client situation

Vaguely written OR not specific to the client situation OR misunderstood managerial impact

Operational impact – clearly considered both positive and negative impact, specific to client situation

Only considered either positive or negative impact, specific to client situation

Vaguely written OR not specific to the client situation OR misunderstood operational impact

Organization Chart [Optional] - This should be labeled “ABC Corporation Organization Chart.” Its purpose is to show the organizational roles of the clients you are working with.

Included – clear, legible org chart highlighting relevant aspects of org  OR 

Not included – detailed comment in Trello with valid justification why org chart was omitted

Included – clear, legible org chart but doesn’t highlight the aspects of the org relevant to project (e.g. can’t distinguish which are unaffected departments) OR Did not link correctly from Mural OR 

Not included – detailed comment with invalid justification why org chart was omitted

Included – org chart unclear and/or illegible OR 

Not included – vague comment in Trello does not sufficiently justify why org chart was omitted

Empathy Map [Optional] – Create an empathy map for your main client/stakeholder in your project.

Included – thorough analysis demonstrated with correct use of quadrants, sketch of client in the middle, and related to project needs OR

Not included – detailed comment in Trello with valid justification why empathy map was omitted

Included – thorough analysis with some mistakes in how the map was used (e.g. missing sketch or incorrect use of quadrants) OR Did not link correctly from Mural OR

Not included – detailed comment with invalid justification why empathy map was omitted

Included – minimal analysis demonstrated and/or misunderstanding of empathy map OR

Not included – vague comment in Trello does not sufficiently justify why empathy map was omitted

As-Is Systems Architecture Checklist [Optional] - Identify the necessary software tools and hardware environment for the as-is system using the provided checklist.

Included – Correctly and thoroughly completed the entire as-is checklist OR

Not included – detailed comment in Trello with valid justification why as-is sys arch was omitted

Included – Completed at least ¾ of the as-is checklist correctly and thoroughly OR Did not link correctly from Mural OR

Not included – detailed comment with invalid justification why as-is sys arch was omitted

Included – Completed less than ¾ of the as-is checklist correctly and thoroughly OR

Not included – vague comment in Trello does not sufficiently justify why as-is sys arch was omitted

Additional To-Be Models or Diagrams [Optional] – Any additional models or diagrams – nottypes” of models/diagrams, just models/diagrams. The models/diagrams you include should help your team better understand the clients’ problems and/or help your client more clearly understand various aspects of your proposed solution.

Included – additional diagrams clearly add value to understanding the current situation, appropriate model types used, followed model type rules/guidelines OR

Not included – detailed comment in Trello with valid justification why additional to-be models were omitted

Included – additional diagrams add some value to understanding the current situation, appropriate model types used, followed most model type rules/guidelines OR Did not link correctly from Mural OR

Not included – detailed comment with invalid justification why additional to-be models were omitted

Included – additional diagrams do not add value to understanding the current situation, inappropriate model types used, or violated model type rules/guidelines OR

Not included – vague comment in Trello does not sufficiently justify why additional to-be models were omitted

Risk Evaluation & Risk Reduction [Optional]

Included – detailed comments specific to the situation provided for each risk, correct interpretation of each risk, all 0 and 1 have actionable risk reduction strategies OR

Not included – detailed comment in Trello with valid justification why risk was omitted

Included – for ¾ or more: detailed comments specific to the situation provided for each risk, correct interpretation of each risk, all 0 and 1 have actionable risk reduction strategies OR Did not link correctly from Mural OR

Not included – detailed comment with invalid justification why risk was omitted

Included – for less than ¾: detailed comments specific to the situation provided for each risk, correct interpretation of each risk, all 0 and 1 have actionable risk reduction strategies or vague comments and/or strategies provided OR

Not included – vague comment in Trello does not sufficiently justify why risk was omitted

To-Be Architecture Checklist [Optional] - Identify the necessary software tools and hardware environment for the as-is system using the provided checklist.

Included – Correctly and thoroughly completed the to-be checklist OR

Not included – detailed comment in Trello with valid justification why to-be arch was omitted

Included – Completed ¾ or more of the to-be checklist correctly and thoroughly OR Did not link correctly from Mural OR

Not included – detailed comment with invalid justification why to-be arch was omitted

Included – Completed less than ¾ of the to-be checklist OR

Not included – vague comment in Trello does not sufficiently justify why to-be arch was omitted

Sample Input/output Documents/Sample Data from Client [Optional] – Include documents provided by your client and any documents you have created or obtained from the client.

Included – Relevant documents and data retrieved from client and stored in Box OR

Not included – detailed comment in Trello with valid justification why sample documents/data were omitted

Not included – detailed comment with invalid justification why sample documents/data were omitted OR vague comment in Trello does not sufficiently justify why documents/data were omitted OR Did not link correctly from Mural