CAISY AI Simulator FAQs

AI simulator content shows to learners as the AI simulator content type. Learners can find this content in multiple ways.

• Enter AI simulator or CAISY in the Search box. The results show all AI simulator content.
• When browsing a channel, look under the Practice tab for AI simulator content. Not all channels have AI simulator content.
• If searching in general, you can use the Type filter, and select to show only AI simulator content if any exists for the keywords searched.
• When scanning the library, look for an area called Artificial intelligence (AI) then the subject Practice with CAISY simulations then the channel of your choosing.

Yes. If learners have the Percipio mobile app, they can launch AI simulator content from the app.

If learners are trying to launch AI simulator content from a browser app on a mobile device, they may need to turn off pop-up blockers for that browser app to play AI simulator content.

AI simulator content is licensed with any one of the Expert collections or the Skillsoft Leadership Development Program. You can check your collection from the License Pools page. Reach out to your Skillsoft account team if you have questions.

Skillsoft provides over 50 scenarios for customers who license any of the Expert collections or the Skillsoft Leadership Development Program (SLDP).

Learners can find this content in multiple ways:

• Enter AI simulator or CAISY in the Search box. The results show all AI simulator content.
• When browsing a channel, look under the Practice tab for AI simulator content. Not all channels have AI simulator content.
• If searching in general, you can use the Type filter, and select to show only AI simulator content if any exists for the keywords searched.
• When scanning the library, look for an area called Artificial intelligence (AI) then the subject Practice with CAISY simulations then the channel of your choosing.

If you license Skillsoft Compliance content and access it directly within the Percipio library, you see 4 compliance simulations under the Legal Compliance area. If you license Skillsoft Compliance content and access it in the Percipio Compliance interface, you will not see the 4 simulations.

A learner can complete up to 50 simulations in a day. The simulations can be all from one topic or from multiple topics.

The CAISY™ AI simulator rates your responses as if it were a human resources representative employing best practices for the given situation.

You will get one of the following ratings:

• Outstanding: You did an excellent job of demonstrating all the behaviors required of the situation, including but not limited to showing empathy where it mattered, addressing the issues at hand, and reaching a conclusion that solved the problem for all involved.
• Acceptable: You may have demonstrated some but not all of the behaviors required of the situation, including showing empathy where it mattered, addressing the issues at hand, and reaching a conclusion that solved the problem for all involved.
• Poor: You demonstrated a lack of familiarity with the behaviors required of the situation. Next time, try showing more empathy where it matters, better addressing the issues at hand, and reaching a conclusion that solves the problem for all involved.

You can copy, paste, and edit the following text in an email if you’d like to invite users to use AI simulator content.

Subject: Announcing a true interactive learning experience!

We are excited to share that you can now practice your skills real-time by responding to AI prompts. Choose the scenario and behavior, then enter one or more responses. The AI simulation shares adaptive feedback based on your responses. You can practice up to 50 different scenarios a day for the most effective learning experience.

To locate the AI simulator content:

1. Search for AI simulator and find the following simulations:
2. Select Launch to start the simulation. You may need to allow browser pop ups to continue.
3. Optionally, select the Scenario mode. You can choose either:
   • Practice: In this mode, you practice the skill presented by the simulator. Use this mode when you want to practice coming up with your own words and phrasing to handle the situation.
   • Role model: In this mode, you allow the AI Simulator to model the skill based on you acting in the opposing role. Use this mode if you are not sure where to begin and want ideas for best practices.
4. Optionally, select Behavior. The behavior you choose guides how the AI simulator responds to your input. By selecting different behaviors, you can practice the same situation and get different AI responses to your input.
5. Read or play the case scenario introduction, then enter a response. You can either type or speak your response. Do not use real names or companies in your responses or any confidential company information.. The AI simulator responds automatically and waits for your response. The AI simulator either continues the conversation asking for additional input from you up to 20 times, or if it deems it has enough input to evaluate your responses before reaching the 20 response limit, it closes the conversation and provides a rating and personalized feedback for future development.

Happy Learning!

There are several factors that can trigger conversation completion.

• When the AI model detects that the conversation has reached a logical conclusion, such as the sale completed.
• When the conversation reaches the 20-turn limit.
• When the learner enters or speaks the word End in the conversation.
• When the leaner selects the End scenario link under the conversation.

If the learner ends the conversation when there are less than 2 messages, they do not see an evaluation.

Yes. You can hide AI simulator scenarios you do not want your learners to see. To hide individual scenarios, follow the instructions for Hiding individual content items.

If you license Skillsoft Compliance content and access it directly within the Percipio library, you see 4 compliance simulations under the Legal Compliance area. If you license Skillsoft Compliance content and access it in the Percipio Compliance interface, you will not see the 4 simulations. Contact your Skillsoft account team if you are interested in using CAISY compliance simulations.

The four compliance simulations located in the Compliance area of the Percipio library are:

• Best Practices for Protecting PII ​
• Delivering Respectful Communication to Employees ​
• Handling a report of Harassment or Discrimination ​
• Strategic Business Activity or a Bribe?

Yes. AI simulator content is represented with the Asset type of Linked content and the Content type of AI simulator.

You can assign and promote AI simulator content. You can also add it to custom channels, journeys, and learning programs and find it in reports with the Asset type filter equal to Linked content and Content type equal to AI simulator.

Yes. When learners use CAISY, you see content under the reporting Asset type filter as Linked content and a Content type of AI simulator. Percipio tracks how long the learner interacts with the content, the start date, and the end date.

The best report to use to see learner activity is the Learner Activity report and filter the Category column for AI Simulator to see simulation starts, completions, and duration for each user who consumed the content. To see aggregated data about each simulation, you can use the Content Access report which shows total unique users who accessed the simulation, total accesses, average duration, and total completions.

Yes. If your LMS supports practice content, you will get the AI simulator content by default in your LMS. Tracking simulation data back to Percipio from the LMS is also supported.

You can provide feedback in two ways: directly to Skillsoft and to your admin.

Directly to Skillsoft:

When you provide feedback to Skillsoft, Skillsoft can improve and develop more AI products, services, and machine-learning technologies. To provide feedback directly to Skillsoft:

• Keep your privacy setting on so Skillsoft can collect your conversation data.
• For every AI simulator response, select the thumbs up icon, . This opens a dialog box for you to rate the response and provide reviewer comments. To submit your feedback, you must acknowledge that Skillsoft will also collect your side of the AI conversation in order to review and learn from it.
• Send an email to ai@skillsoft.com. Here are some things to consider for feedback:
  • What worked well / didn’t work well?
  • Was the user experience intuitive? Delightful?
  • How effective is this for you as an interactive teaching tool?
  • How could it be better?
  • What other kinds of interactive scenarios would you like to see?
  • How would you expect to use this at your organization?

To your admins:

To provide feedback to your admins, select the Give scenario feedback link at the end of the scenario, if enabled for your site. By using the Learner feedback dashboard, they can evaluate your anonymous feedback with others' feedback to understand the effectiveness of CAISY in your organization

• We do not collect IP addresses, location data, or any personal data from the learner unless the learner inputs it as part of their conversations. Learn more about how we protect your privacy by reviewing our Privacy Notice - Skillsoft.
• All data entered is encrypted in transit and at rest.
• The information that the learner enters into CAISY passes to Microsoft Azure OpenAI so that the user can continue the conversation and receive adaptive, personalized feedback.
• Microsoft Azure OpenAI does not store any conversation data. Skillsoft has MS Azure OpenAI GPT prompt moderation exception so no GPT inputs or outputs are logged or reviewed by Microsoft. Similarly, no data from CAISY is available for usage to train any models.
• Skillsoft stores AI conversation data in authenticated and encrypted AWS storage.
• Skillsoft logs the AI simulator side of the conversation, time stamps, AI options used during the conversation, ratings, feedback, if guardrails are triggered, and other non-user specific data for reporting and analytics.
• Skillsoft only logs the learner side of the conversation if the setting: Allow Skillsoft to collect, store, and analyze my conversations in order to improve the AI Simulator is enabled. Admins can make this setting on or off by default but the learner can change it for themselves unless the admin restricts it.
• Learners can opt out of having their side of the conversation logged by Skillsoft:
1. With the AI simulator content open, select Settings.
2. Turn off: Allow Skillsoft to collect, store, and analyze my conversations in order to improve the AI Simulator This setting is on by default for companies using the US data center and off by default for companies using the EU data center. Admins can change the default behavior of this setting for the learner.
3. Select Save.
• If the user selects the thumbs up icon to an AI generated response and provides feedback, they must acknowledge that their side of the conversation will be shared with Skillsoft even if they turned off the setting: Allow Skillsoft to collect, store, and analyze my conversations in order to improve the AI Simulator. If the admin restricts it, learners cannot override this setting.
• Any data that the learner shares helps Skillsoft provide, improve, and develop AI products, services, and machine-learning technologies. For example, we use your feedback to increase the effectiveness of CAISY safety policies and help minimize some of the challenges inherent to large language models.  
• Skillsoft has a process for improving CAISY, which includes review of conversations reported as being low quality. This helps us identify common types of problematic responses and how CAISY might improve. Only limited Skillsoft employees can access and review conversation data.
• We have concurrent GPT prompting for every user input to test for inappropriate inputs. If we discover inappropriate inputs, we stop the conversation and inform the user. GPT also has it's own guardrails to flag inappropriate input which we also monitor. If we discover inappropriate input from that perspective, we also stop the conversation and inform the user.

Microsoft Azure does not log or store any conversation data. Skillsoft only stores conversation data in authenticated and encrypted AWS storage. Only limited Skillsoft employees can access and review conversation data.

Skillsoft stores this data per the terms in your contract. Skillsoft can delete CAISY log data upon request.

No. CAISY does not save audio recordings. Audio is only in memory until it is transcribed to text and then the audio is immediately discarded.

We have concurrent GPT prompting for every user input to test for inappropriate inputs. If we discover inappropriate inputs, we stop the conversation and inform the user. GPT also has it's own guardrails to flag inappropriate input which we also monitor. If we discover inappropriate input from that perspective, we also stop the conversation and inform the user.

• The information that the learner enters into CAISY passes to Microsoft Azure OpenAI so that the user can continue the conversation and receive adaptive, personalized feedback.
• Microsoft Azure OpenAI does not store any conversation data. Skillsoft has MS Azure OpenAI GPT prompt moderation exception so no GPT inputs or outputs are logged or reviewed by Microsoft. Similarly, no data from CAISY is available for usage to train any models.
• Skillsoft stores AI conversation data in authenticated and encrypted AWS storage.
• Skillsoft logs the AI simulator side of the conversation, time stamps, AI options used during the conversation, ratings, feedback, if guardrails are triggered, and other non-user specific data for reporting and analytics.
• Skillsoft only logs the learner side of the conversation if the setting: Allow Skillsoft to collect, store, and analyze my conversations in order to improve the AI Simulator is enabled. Admins can make this setting on or off by default but the learner can change it for themselves unless the admin restricts it.
• Learners can opt out of having their side of the conversation logged by Skillsoft:
1. With the AI simulator content open, select Settings.
2. Turn off: Allow Skillsoft to collect, store, and analyze my conversations in order to improve the AI Simulator This setting is on by default for companies using the US data center and off by default for companies using the EU data center. Admins can change the default behavior of this setting for the learner.
3. Select Save.

It may be possible upon request. Reach out to our Support team for assistance.

CAISY provides subjective analysis of conversations. It is not answering specific questions or stating facts. Extensive testing has not revealed significant issues with outputs. Users are warned that responses might not be accurate, and they should use their own judgment when reviewing the response.

Each scenario has a detailed description that includes guidance which guards the conversation.

CAISY provides subjective analysis of conversations. It is not answering specific questions or stating facts. Extensive testing has not revealed significant issues with outputs. Users are warned that responses might not be accurate, and they should use their own judgment when reviewing the response.

While it is not zero, it is low and extensive testing has not revealed issues that have raised concerns.

It is improbable CAISY could produce any hallucinations or bias based on demographics because CAISY doesn't capture any demographics information, such as country, gender, or age, and all learners use the same AI model version.

Extensive Quality Assurance testing has not revealed significant issues with outputs. This simulator provides subjective analysis of conversations. It is not answering specific questions or stating facts. Each scenario has a detailed description that includes guidance which guards the conversation.

CAISY utilizes the built-in Azure OpenAI guards and adds additional guards which review and evaluate the user inputs for off-topic, abuse, and misuse. If these are detected, the user will be alerted, and the conversation will be halted and require editing or restarting.

Users are warned that responses might not be accurate. Users must acknowledge prior to using CAISY that they should use their own best judgment.

CAISY provides subjective analysis of conversations. It is not answering specific questions or stating facts. Extensive testing has not revealed significant issues with outputs. Users are warned that responses might not be accurate, and they should use their own judgment when reviewing the response.

Please submit feedback so we can review and improve any rare mistake that may occur.

CAISY leverages hallucinations to create variety in the conversation. Extensive testing indicates the conversations stay on topic and feedback is appropriate.

CAISY is a feature of Skillsoft Percipio, which conforms to WCAG 2.1 A and AA accessibility standards with some exceptions noted in its VPAT Conformance Report.

Note: All non-conformances related to the Conversation AI Simulator (CAISY) have been remediated as of February 2024.

CAISY uses generative AI in an ethical and responsible way through the implementation of appropriate guardrails, widely recognized as important for AI systems like ChatGPT.

CAISY is developed and hosted on Microsoft Azure OpenAI and does not use OpenAI 's ChatGPT consumer product. Microsoft Azure OpenAI uses the same models as OpenAI but with the security capabilities of Microsoft Azure:

• Encryption: Azure OpenAI automatically encrypts the user's data. The encryption protects their data and helps customers meet organizational security and compliance commitments.
• Data Security: Azure OpenAI offers us the ability to do regional deployments to ensure EU hosted data is processed in EU and North America hosted customers data is processed within North America.
• Azure OpenAI Limited Access Review: Modified Abuse Monitoring - Microsoft Azure does not log or review any OpenAI conversation (inputs/outputs). Skillsoft only logs conversations based on opt-in/out or explicit sharing when reporting feedback within the AI simulator.

Additional security layers help to mitigate biases in training data, promote fairness, reduce the spread of false information, and prevent offensive or inappropriate responses through the conversations that learners have with the AI trainers.

Every time a conversational scenario takes place, the system is monitoring for abuse, correct context, and scenario accuracy. For example, if a learner were to initiate an off-topic or inappropriate conversation with CAISY, the simulation would guide the learner to only make appropriate statements. The learner can then resume or restart the conversation. Striking this balance is crucial to avoid overly restrictive measures while maintaining responsible deployment.

The data is encrypted within the FedRAMP authorized boundary where authenticated access is required.

The current models that are used are Commercial off-the-shelf (COTS) GPT models created by MS Azure OpenAI, GPT-3.5 and 4 models. They do not list all the data that was used for training.

CAISY utilizes multiple custom prompts that act in parallel to conduct, guide, guard, rate, and evaluate the conversations. We monitor feedback and are constantly QA testing the system.

No, we do not train or fine-tune AI models used by CAISY with any user inputs.

No, but it does use OpenAI models of which the core foundation model is similar to ChatGPT.

CAISY is developed and hosted on Microsoft Azure OpenAI and does not use OpenAI 's ChatGPT consumer product. Microsoft Azure OpenAI uses the same models as OpenAI but with the security capabilities of Microsoft Azure:

• Encryption: Azure OpenAI automatically encrypts the user's data. The encryption protects their data and helps customers meet organizational security and compliance commitments.
• Data Security: Azure OpenAI offers us the ability to do regional deployments to ensure EU hosted data is processed in EU and North America hosted customers data is processed within North America.
• Azure OpenAI Limited Access Review: Modified Abuse Monitoring - Microsoft Azure does not log or review any OpenAI conversation (inputs/outputs). Skillsoft only logs conversations based on opt-in/out or explicit sharing when reporting feedback within the AI simulator.

Percipio is a multi-tenant system where all tenants share the same services and resources. Customer data is isolated with tenant keys and/or S3 buckets.

Yes. Skillsoft Percipio has been FedRAMP authorized since March 4, 2022. A Significant Change Request(SCR) for the technologies supporting the CAISY feature of Percipio was signed by our Third Party Assessment Organization (3PAO), Booz Allen, on January 18, 2024 following a review by our FedRAMP sponsor, USDA. CAISY uses AI services from the US Microsoft Commercial Cloud, which is also FedRAMP authorized.