Technical FAQs for AI Coaching Guide

Any data that you share helps Skillsoft improve the service. For example, we use your feedback to increase the effectiveness of AI safety policies.

Directly to Skillsoft:

To provide feedback directly to Skillsoft:

• Send an email to ai@skillsoft.com. Here are some things to consider for feedback:
  • What worked well / didn’t work well?
  • Was the user experience intuitive? Delightful?
  • How effective is this for you as an interactive teaching tool?
  • How could it be better?
  • What other kinds of interactive scenarios would you like to see?
  • How would you expect to use this at your organization?

AI Coaching Guide is accessible and will be added into the updated VPAT coming summer 2024.

does not store user IP addresses, nor does it collect location data, or any personal information (PII) from the learner. Skillsoft network security separately logs IP addresses for the purposes of addressing security risks, such as Distributed Denial-of-Service (DDOS) attacks and intrusion detection. Learn more about how we protect your privacy by reviewing our Privacy Notice - Skillsoft.

Skillsoft always logs the AI side of the conversation, time stamps, AI options used during the conversation, ratings, feedback, if guardrails are triggered, and other non-user specific data for reporting and analytics.

Skillsoft only logs the learner side of the conversation if the setting: Allow Skillsoft to collect, store, and analyze my conversations to improve the AI guide. Admins can adjust this setting, turning it on or off by default for their learners, ask your CSM to update the settings. The learner can change it for themselves unless the admin restricts it.

Learners can opt out of having their side of the conversation logged by Skillsoft, even if it is enabled for your company:

1. Open the AI chat.

2. Turn off: Allow Skillsoft to collect, store, and analyze my conversations in order to improve the AI Coach..

• All data entered is encrypted in transit and at rest.
• The information that the learner enters into the AI Coaching Guide passes to Microsoft Azure OpenAI so that the user can continue the conversation and receive dynamic, personalized feedback.
• Microsoft Azure OpenAI does not store any conversation data. Skillsoft has MS Azure OpenAI prompt moderation exception so no Generative AI inputs or outputs are logged or reviewed by Microsoft. Similarly, no data from the AI Coaching Guide is available for usage to train any models.
• Skillsoft stores AI conversation data in authenticated and encrypted AWS storage.

• Feedback learners submit via the thumbs up icon is logged in the conversation logs.

  

  The feedback is periodically reviewed by the Product teams for sentiment, requested improvements, and potential issues. Appropriate enhancements or improvements are considered and prioritized.

• Any data that you share helps Skillsoft improve the service. For example, we use your feedback to increase the effectiveness of AI safety policies.
• Skillsoft has a process for improving the AI Coaching Guide, which includes review of conversations reported as being low quality. This helps us identify common types of problematic responses and how the AI Coaching Guide might improve.
• Only limited Skillsoft employees can access and review conversation data.

Yes. All data entered is encrypted in transit and at rest.

Microsoft Azure does not log or store any conversation data. Skillsoft only stores conversation data in authenticated and encrypted AWS storage. Only limited Skillsoft employees can access and review conversation data.

Skillsoft stores this data per the terms in your contract. Skillsoft can delete the AI Coaching Guide log data upon request.

No. The AI Coaching Guide does not save audio recordings. Audio is only in memory until it is transcribed to text and then the audio is immediately discarded. Your data remains yours. Your audio input and transcription data aren't logged during audio processing.

• The information that the learner enters into the AI Coaching Guide passes to Microsoft Azure OpenAI so that the user can continue the conversation and receive dynamic, personalized feedback.
• Microsoft Azure OpenAI does not store any conversation data. Skillsoft has MS Azure OpenAI prompt moderation exception so no Generative AI inputs or outputs are logged or reviewed by Microsoft. Similarly, no data from the AI Coaching Guide is available for usage to train any models.
• Skillsoft stores AI conversation data in authenticated and encrypted AWS storage.

Yes, it is possible to delete per user and per organization upon request. Reach out to our Support team for assistance.

We have concurrent GenAI model prompting for every user input to test for inappropriate inputs. If we discover inappropriate inputs, we stop the conversation and inform the learner. The GenAI model also has it's own guardrails to flag inappropriate input which we also monitor. If we discover inappropriate input from that perspective, we also stop the conversation and inform the user.

GPT has guardrails in place to flag and stop inappropriate outputs.

We have concurrent GenAI model prompting for every user input to test for inappropriate inputs. If we discover inappropriate inputs, we stop the conversation and inform the learner. The GenAI model also has it's own guardrails to flag inappropriate input which we also monitor. If we discover inappropriate input from that perspective, we also stop the conversation and inform the user.

The AI Coaching Guide provides subjective analysis of conversations. It is not answering specific questions or stating facts. Extensive testing has not revealed significant issues with outputs. Users are warned that responses might not be accurate, and they should use their own judgment when reviewing the response.

The AI Coaching Guide provides subjective analysis of conversations. It is not answering specific questions or stating facts. Extensive testing has not revealed significant issues with outputs. Users are warned that responses might not be accurate, and they should use their own judgment when reviewing the response.

While it is not zero, it is low and extensive testing has not revealed issues that have raised concerns.

It is improbable the AI Coaching Guide could produce any hallucinations or bias based on demographics because the AI Coaching Guide doesn't capture any demographics information, such as country, gender, or age, and all learners use the same AI model version.

Extensive Quality Assurance testing has not revealed significant issues with outputs. The AI Coaching Guide offers guidance and support throughout the coaching process. While it provides answers to relevant questions, it does not state facts. Each prompt has a detailed description that includes guidance which guards the conversation.

The AI Coaching Guide utilizes the built-in Azure OpenAI guards and adds additional guards which review and evaluate the user inputs for off-topic, abuse, and misuse. If these are detected, the user will be alerted, and the conversation will be halted and require editing or restarting.

Users are warned that responses might not be accurate. Users must acknowledge prior to using the AI Coaching Guide that they should use their own best judgment.

The AI Coaching Guide provides subjective analysis of conversations. It is not answering specific questions or stating facts. Extensive testing has not revealed significant issues with outputs. Users are warned that responses might not be accurate, and they should use their own judgment when reviewing the response.

Please submit feedback so we can review and improve any rare mistake that may occur.

The AI Coaching Guide leverages non-zero temperature to create variety in the conversation. Extensive testing indicates the conversations stay on topic and feedback is appropriate.

the AI Coaching Guide uses generative AI in an ethical and responsible way through the implementation of appropriate guardrails, widely recognized as important for AI systems like ChatGPT.

In addition, the AI Coaching Guide uses MS Azure OpenAI default content filtering and adds additional tests and guards to detect and minimize inputs that might stimulate bias or other inappropriate responses.

The AI Coaching Guide is developed and hosted on Microsoft Azure OpenAI and does not use OpenAI 's ChatGPT consumer product. Microsoft Azure OpenAI uses the same models as OpenAI but with the security capabilities of Microsoft Azure:

• Encryption: Azure OpenAI automatically encrypts the user's data. The encryption protects their data and helps customers meet organizational security and compliance commitments.
• Data Security: Azure OpenAI offers us the ability to do regional deployments to ensure EU hosted data is processed in EU and North America hosted customers data is processed within North America.
• Azure OpenAI Limited Access Review: Modified Abuse Monitoring - Microsoft Azure does not log or review any OpenAI conversation (inputs/outputs). Skillsoft only logs conversations based on opt-in/out or explicit sharing when reporting feedback within the AI simulator.

Additional security layers help to mitigate biases in training data, promote fairness, reduce the spread of false information, and prevent offensive or inappropriate responses through the interactions that learners have with the AI Coaching Guide.

Every time a conversational scenario takes place, the system is monitoring for abuse, correct context, and scenario accuracy. For example, if a learner were to initiate an off-topic or inappropriate conversation with the AI Coaching Guide, the guide would guide the learner to only make appropriate statements. The learner can then resume or restart the conversation. Striking this balance is crucial to avoid overly restrictive measures while maintaining responsible deployment.

Data is encrypted in flight with TLS 1.2+ and encrypted at rest per FIPS 140-2.

The data is persisted within the FedRAMP authorization boundary where limited and authenticated access is required.

The current models that are used are Commercial off-the-shelf (COTS) OpenAI models. The models are created by OpenAI and hosted by MS on Azure with enhanced data security and content filtering for safety. They do not list all the data that was used for training.

The AI Coaching Guide utilizes multiple custom prompts that act in parallel to conduct, guide, and guard the conversations. We monitor feedback and periodically QA test the system.

No, we do not train or fine-tune AI models used by the AI Coaching Guide with any user inputs.

No, but it does use OpenAI models of which the core foundation model is similar to ChatGPT.

The AI Coaching Guide is developed and hosted on Microsoft Azure OpenAI and does not use OpenAI 's ChatGPT consumer product. Microsoft Azure OpenAI uses the same models as OpenAI but with the security capabilities of Microsoft Azure:

• Encryption: Azure OpenAI automatically encrypts the user's data. The encryption protects their data and helps customers meet organizational security and compliance commitments.
• Data Security: Azure OpenAI offers us the ability to do regional deployments to ensure EU hosted data is processed in EU and North America hosted customers data is processed within North America.
• Azure OpenAI Limited Access Review: Modified Abuse Monitoring - Microsoft Azure does not log or review any OpenAI conversation (inputs/outputs). Skillsoft only logs conversations based on opt-in/out or explicit sharing when reporting feedback within the AI simulator.

Skillsoft Coaching implements the principle of least privilege to ensure users only have the minimum necessary access. We assign roles and permission based on organizational boundaries and ensures that roles are strictly enforced and regularly audited. We implement strict access control policies that define who can access what data and under what conditions.