CAISY™ AI Simulation Dashboard

The CAISY™ AI Simulation dashboard shows a visual summary of how your learners are interacting with CAISY simulations. You see a summary of total accesses as well as data broken down by simulation.

Dashboard Insights

This dashboard gives you insights into the most popular CAISY simulations on your platform and how learners are scoring on these simulations. You can:

  • See which and how many CAISYs learners taking.
  • See how many learners are engaged in learning skills through CAISY.
  • Gain insights about which behaviors and modes learners are engaging with.
  • Check which are the top 10 CAISY scenarios that learners are interacting with. Use this information to reflect on the skills that learners are focusing their time on.
  • See how learners are scoring on the proficiency scale. Consider assigning additional training to help build up weaker skills.
  • Gain insight into how many learners are practicing these scenarios more than once and how much time are they spending. Encourage users to continue practicing in order to improve their proficiency levels on certain skills.

Get Started

To access this dashboard and customize it for your needs:

  1. From the left navigation bar, select Analytics. Switch to the All dashboards tab.
  2. Select CAISY AI Simulation.The dashboard page displays.
  3. To see just the data you need, specify global filters for this dashboard. You find all filters under the Filters drop down.
  4. Once you specify filters, select Apply filters to update the dashboard.
  5. Select Load default filters, then Apply filters to reset all filters to default.

Filters

The following filters are available for this dashboard:

  • Simulation type: Conversation or Presentation. By default both are selected. In support of early access customers of LX Design Studio, you now see a checkbox to show CAISY for Presentations on the dashboard. For non-early access customers, you see the checkbox, but checking it does not change the dashboard display.
  • Date Range: Filters the data by a specified range of dates that you select from the drop-down list. You can also select a custom range from the beginning of one month to the end of the same month or the end of another month.
    • Start: Instead of using a Date Range preset, use this to specify a start date. When you enter a Start date, the Date Range Presets filter changes to Custom. The platform started recording CAISY activity on March 1, 2024. Activity prior to that date for CAISY activity is unavailable.
    • End: Instead of using a Date Range preset, use this to specify an end date. When you enter an End date, the Date Range Presets filter changes to Custom.
  • Audience: Predefined group of users. Enter the name of an audience and then select it from the list. You can display the data for multiple audiences at the same time by entering more than one audience name. You can select up to 20 audiences.
  • CAISY title: The title of the AI scenario. Use this filter if you want to just review specific scenarios. The scenarios you select here impact what you see on the rest of the page.
  • Scenario category: AI simulations are grouped by category. If you want to see AI simulations for one or more categories, select those categories from the list. What you select here impacts what you see on the rest of the page.
  • Include Inactive users: Select if you want to include inactive users in your counts. If you include inactive users, you may not get an accurate picture of your organization current proficiency levels specially if those inactive users left your company.

About this Dashboard

With this dashboard, you can gauge where your learners are on the proficiency spectrum for each CASIY scenario at any given point in time within the specified date range. The dashboard only reports on the best proficiency rating of each learner within that date range if they have taken the scenario more than once.

For example, this dashboard helps you see how you might need to tailor programs based on results. If many users rank in the Poor category for a specific simulation, you might consider assigning additional contextual content to support the skill the simulation is teaching. Conversely, if many learners score in the Outstanding category, and only a few score in the Poor category, you can tailor programs just for learners scoring poorly, while assigning learners who score outstanding to the next highest skill set or challenging them with a different simulation behavior.

If you haven't yet assigned CAISY scenarios or added them to custom journeys or learning programs, you can see what scenarios your learners are focusing on and if they align to your corporate business objectives. If you configured audiences for department and job role, you can get a sense of the proficiency level of different departments or job roles by using the audience filter to see the scenarios accessed and the proficiency levels achieved.

This dashboard contains the following sections:

  1. In this section you see:

    • Total simulations completed: The total number of simulations completed in the specified date range. If one learner completed a single simulation 8 times, each time the learner took the simulation is added to the overall count, in this case 8. If one learner took 6 different simulations one time each, the count goes up by 6.
    • Unique simulations completed:The number of unique simulations completed in the specified date range. If one learner completed a single simulation multiple times, that is only counted as one unique simulation. If multiple users completed the same simulation, that is only counted as one unique simulation.
    • Unique learners completed: The number of unique learners who completed a simulation in the specified date range. If one learner completed more than one simulation or one simulation multiple times, they are only counted once.

  2. In this section you see a list of the top 10, most-accessed AI scenarios based on the filters set. The list is initially sorted by the highest number of unique learners who accessed each scenario. You can sort by either of the columns.

    For each scenario you see:

    • SCENARIO TITLE: The name of each AI simulation. If you select the name, you can view additional details of that one scenario.
    • PROVIDER NAME: This helps you distinguish if the CAISY scenario is a conversation or a presentation. You build CAISY for presentations in LX Design Studio.
    • Unique learners: The number of unique learners within the specified date range who completed each scenario title. For the colored distribution bar, if the same learner completed a scenario multiple times each time using a different behavior, they are also counted for each behavior they took. If they took the same behavior multiple times, that is only counted once. This is why the bar total may not match the Unique learners total.
    • Score range: Of all the unique learners who completed the scenario title within the date range, you see the breakdown by level of each users' highest proficiency level. Levels mean the following for the learners:
      • Novice (0-9 points): Novice means that you are starting with very limited familiarity on the topics critical for the skill.
      • Aspiring (10-49 points): Aspiring means that you have demonstrated minimal capabilities for the specific skill and should make an investment in learning to achieve proficiency.
      • Developing (50-69 points): Developing means that you have some proficiency with the specific skill and only need to close a few knowledge gaps to achieve proficiency.
      • Proficient (70-89 points): Proficiency means that you have met the criteria for applying your knowledge with the skill to practical challenges in your work. We recommend you strive to achieve a minimum level of Proficiency for the skills you are developing.
      • Advanced (90-100 points): Advanced means that you demonstrate a thorough understanding of the skill and may be a candidate to mentor other learners within your organization.

    You can select the name of the scenario to get more details.

    The detailed CAISY AI Simulation page includes the following information:

    • Date and audience filters: Initially carried over from the main dashboard, you can change them. What you specify here impacts the results shown on the page.
    • Unique learners: The number of unique learners within the specified date range who completed each scenario title.
    • Average skill level:For the date and audience selected, you see how many times those unique learners completed the simulations and how each completion was rated: Outstanding, Acceptable or Poor. Lets assume 201 scenarios were completed and of which 70 got a rating of Outstanding, 120 got Acceptable, and 40 got Poor. Then percentages calculation for each category. Outstanding: (70/201)*100= 35% Acceptable: (120/201)*100= 60% Poor: (40/201)*100= 20%. Unrated items are those taken in Role model mode.
    • Results distribution by CAISY behavior: When learners launch a scenario, they choose whether they want to interact in Role model or Practice mode. If they choose Practice mode, they can then choose which behavior they want to interact with: Defensive, Dismissive or Accepting. In this box, you see all completions grouped by mode and behavior and the proficiency level for each. When learners choose the Role model mode, the platform does not give a rating because the learner is observing CAISY during the simulation about how best to solve the problem. The platform does show how many learners took the CAISY in role model mode. The data shown here is based on the filter set.
    • Simulation results table: This table gives you the detail of every user who completed the scenario, the mode and behavior they used, and their proficiency level each time they completed the scenario.

      The default columns selected for this table are:

      NOTE: If you are not able to see the user detail columns, ask your admin for permissions.
      • USER ID: The value entered in the User ID field. It is used to identify a user inside the platform.
      • LAST NAME: The value entered in the Last name user attribute. Usually the user's last name.
      • FIRST NAME: The value entered in the First name user attribute. Usually the user's first name.
      • SCENARIO MODE:The mode of the AI simulation: role model or practice.
      • SCENARIO BEHAVIOR: When using an AI simulation in practice mode, this is the behavior you are practicing against: dismissive, defensive, or accepting.
      • # OF RECURRENCES: The number of times a learner returned and re-completed the content.
      • FIRST ATTEMPT RATING:The rating given the first time the learner completed the item. If the CAISY was taken in role model mode, this column is left blank.
      • LAST ATTEMPT RATING:The rating given the last time the learner completed the item. If the CAISY was taken in role model mode, this column is left blank.
      • BEST SCORE:The best score for the learner within the date range specified.

      Additional columns that you can select for this table are:

      • LOGIN NAME: The value entered in the Login name attribute for the user.

      • EMAIL ADDRESS: The value entered in the Email address user attribute; the user's email address.

      • USER STATUS: The status of the user in the platform. Valid values are active or inactive.

      • AUDIENCES: Shows the audiences where the learner is a member.

      • FIRST ATTEMPT RATING DATE:The date the first rating was given.

      • LAST ATTEMPT RATING DATE:The date the last rating was given.

      • ESTIMATED DURATION (SECONDS): Total estimated amount of time it takes a learner to complete the specified content. Time is listed in total seconds.

      • LEARNING HOURS (SECONDS): The total number of hours a learner spent in content learning. If the item is a Skillsoft course, the total includes time spent watching videos while in the course (not the time spent on the same videos if launched in video-only mode) and working on knowledge checks within the course. If you see learners with a completed status and with 0 duration, the learner only took the test and did not watch any of the content. Time is listed in total seconds.

      • User Attributes (varies): These fields are defined by (and are unique to) your organization. For more information, see User Attributes. If you are using an attribute type of Skillsoft Platform User, the value in the field can be either first and last name, email address, login ID, or external user ID. The value that appears depends on what has been specified for the individual user.

      From this table, you can:

      • Download report: download the entire list based on filters set into a CSV format that you can use for further data analysis.
      • Customize your view by sorting, filtering, ordering and choosing columns of data to show. For instructions, see Organize columns in tables.

  3. In this section you see a list of all AI simulations and their corresponding data based on the filters set. With this list you can:

    • Download report: download the entire list based on filters set into a CSV format that you can use for further data analysis.
    • Customize your view by sorting, filtering, ordering and choosing columns of data to show. For instructions, see Organize columns in tables.

    The columns available for this section include:

    • SCENARIO TITLE: The name of each AI simulation. If you select the name, you can view additional details of that one scenario.
    • PROVIDER NAME: This helps you distinguish if the CAISY scenario is a conversation or a presentation. You build CAISY for presentations in LX Design Studio.
    • SCENARIOS COMPLETED: The total number of times the scenario was completed. If one learner completed it 3 times, the count would be 3.
    • UNIQUE LEARNERS: The number of unique learners who completed the scenario at least once.
    • ESTIMATED DURATION (SECONDS): Total estimated amount of time it takes a learner to complete the specified content. Time is listed in total seconds.
    • LEARNING HOURS (SECONDS): The total number of hours a learner spent in content learning. If the item is a Skillsoft course, the total includes time spent watching videos while in the course (not the time spent on the same videos if launched in video-only mode) and working on knowledge checks within the course. If you see learners with a completed status and with 0 duration, the learner only took the test and did not watch any of the content. Time is listed in total seconds.
    • AVG. PROFICIENCY LEVEL: Shows the average proficiency level across all learners achieved for that CAISY.

    • CONTENT TYPE: The type of content. Content types can be subsets of asset types. Content types are equivalent to the asset type for courses, videos, books, audiobooks, audio summaries, book summaries, and journeys. When an asset type is set for assessment, you can see a content type of Skill Benchmark. When an asset type is set for linked content, you can see content types for AI simulator, cheatsheets, interactive courses, test preps, labs, bootcamps, partner content, and any custom content items you added. When an asset type is set for scheduled content, you see content types of live courses and live events.