Guardian: A Solution for Responsible

Generative AI Use

Introducing Guardian, a powerful solution designed to ensure responsible use of Generative AI, particularly OpenAI's
ChatGPT, within organizations. Guardian provides a comprehensive risk management framework to proactively
identify and prevent potential harm associated with Generative AI, ensuring alignment with organizational policies and
promoting ethical AI adoption.

(Team Name - One-Man Wolfpack)

(Team size - 1)

(Team Member name - Tushar Harbola)

Introduction

The growing adoption of Generative AI, especially ChatGPT, in organizations presents exciting opportunities
alongside the crucial responsibility of ensuring its ethical and responsible use. However, there's a lack of effective
systems to monitor and control AI usage for risk mitigation and compliance. This creates a critical need for a solution
like Guardian to address potential risks and promote responsible adoption of AI.

Potential Benefits of AI

Generative AI offers significant potential benefits for organizations, including increased productivity, automation of
repetitive tasks, and innovative problem-solving.

Importance of Responsible Use

Ensuring the responsible use of AI is crucial to prevent potential misuse, ethical violations, and legal complications.

Problem Statement

Organizations currently lack a system to effectively monitor and control AI usage, leading to potential risks related to
policy violations and harmful user interactions.
Proposed Approach

To tackle the problem of unmonitored Generative AI use, and unlock its full potential, we propose the implementation
of Guardian, a proxy-based real-time monitoring and risk management system.

Guardian acts as a proxy, monitoring both the user's prompt and ChatGPT's response. Based on the 'risk score' of the
prompt or the reply, Guardian takes appropriate actions to ensure responsible use of generative AI.

Guardian's approach combines innovative functionalities to ensure responsible Generative AI use and mitigate
associated risks effectively. The system achieves automated rule generation, real-time monitoring, risk-based
alerting, and a high degree of admin control for comprehensive AI risk management.

Here's the high-level overview of the working of Guardian:

1. User Interaction: When an employee interacts with ChatGPT by entering a prompt, this interaction is intercepted
by Guardian. This could be implemented by routing all requests through Guardian, which acts as a proxy server.

2. Real-time Monitoring: Guardian analyzes the prompt and the generated response from ChatGPT in real-time. It
uses natural language processing (NLP) techniques to understand the content and context of these interactions.
This could involve checking for specific keywords, phrases, or patterns that might indicate a potential policy
violation or harmful content generation.

3. Risk Assessment: Based on the analysis, Guardian assigns a risk score to the interaction. This score is calculated
using a machine learning model that has been trained on a dataset of policy-violating and non-violating
interactions. The model could use features such as the presence of certain keywords, the sentiment of the text,
and other relevant factors.

4. Preventative Measures: Depending on the risk score, Guardian takes appropriate actions. For low-risk
interactions, it might inject a warning message into the ChatGPT response. This message could remind the user of
relevant company policies and encourage responsible AI use. For high-risk interactions, Guardian could block the
response entirely and require the user to provide additional verification, such as entering a CAPTCHA or confirming
their identity. In extreme cases, Guardian could temporarily limit the user’s access to ChatGPT.
5. Transparency and Control: All user interactions, risk scores, and actions taken by Guardian are logged in a
database. This database can be accessed through a secure dashboard, providing you and your management team
with oversight and control. You can review the monitoring rules and risk thresholds, and adjust them as needed. You
can also view detailed reports on user interactions and the actions taken by Guardian.
Core Functionalities in Depth

Let's delve deeper into the key functionalities provided by Guardian, specifically automated rule generation and real-
time monitoring with minimal latency. Understanding these core aspects is essential to grasp the system's
comprehensive approach to risk management and responsible Generative AI use within organizations.

Automated Rule Generation

Guardian will leverage Azure AI Language services for NLP tasks to scan documents containing organizational
policies, laws, and regulations. The service’s entity recognition and key phrase extraction features will be used to
identify important terms and concepts that will form the basis of the monitoring rules. These rules will then be used to
identify risky behavior and responses, ensuring compliance and risk mitigation.

Real-time Monitoring

Real-time monitoring refers to the ongoing process of scrutinizing and interpreting the interactions between the user
and the AI model as they transpire. In the context of Guardian, this involves the real-time analysis of both the user’s
prompts and the AI model’s responses. This is a critical function for promptly identifying potential risks or policy
violations, thereby enabling immediate corrective action.

To facilitate real-time monitoring, Guardian operates as a streamlined proxy server, intercepting all exchanges
between the user and the AI model. It employs Natural Language Processing (NLP) techniques to comprehend the
substance and context of these interactions, scanning for specific keywords, phrases, or patterns that could signify a
potential policy violation or the generation of harmful content."
Workflow of Guardian
Risk Management

Guardian introduces risk thresholds for monitoring rules and scores interactions for potential risk. It empowers
administrators to fine-tune alerts and prevention measures based on these scores and thresholds, ensuring a
comprehensive and adaptive approach to risk management and responsible AI usage within organizations.

Risk Thresholds

By setting risk thresholds, Guardian enables organizations to define the level of risk they are willing to tolerate in their AI
systems. These thresholds serve as the basis for monitoring rules and help identify potential risks associated with AI-
generated content. By leveraging Guardian's risk thresholds, organizations can proactively mitigate risks and ensure
responsible AI use.

Fine-tuning Alerts

Guardian adjusts prevention measures based on risk scores and specific organizational requirements. By analyzing
risk scores generated by monitoring rules and considering the unique needs of each organization, Guardian can tailor
its prevention measures to ensure responsible generative AI use.

With the ability to adjust prevention measures, organizations can strike the right balance between enabling AI
innovation and mitigating potential risks. By customizing prevention measures, organizations can align AI systems with
their specific requirements and ensure that AI-generated content meets their desired standards of quality, ethics, and
safety.
Admin Control and Customization

Guardian provides administrators with full control over the system, empowering them to review and approve
monitoring rules, adjust risk thresholds, and customize prevention actions based on specific organizational needs. This
level of control and customization is essential for effective implementation and alignment with organizational policies.

Review and Approval

Admins review and approve generated rules, ensuring their alignment with organizational policies.

Customization

Adjusts alerting and prevention rules based on risk thresholds and specific organizational requirements.
Sample Risk Indicators

Here are some examples of risky user behavior and harmful AI responses that Guardian is designed to address. These
indicators emphasize the potential harm within organizations and underscore the importance of having a robust
solution like Guardian to mitigate risk effectively.

Risky User Behavior

Guardian helps organizations address a wide range of risks associated with generative AI use. Some of the risks that
Guardian aims to mitigate include violations of organizational policies, expressions of violence, and unauthorized
access to personal or copyrighted information.

Harmful AI Responses

Guardian aims to help organizations identify and address a wide range of risks associated with generative AI use.
Some of the risks that Guardian can help mitigate include insider trading, recruitment, malware development, and data
manipulation.
Building the Prototype

Requirement Gathering

We’ll start by understanding the project requirements in detail. This includes understanding the safety concerns
associated with large language models, the need for real-time monitoring, and the desired functionalities of Guardian.

Design and Wireframing

We’ll create a visual representation of the prototype using wireframes. This will outline the user interface of the admin
dashboard, the interaction flow between the user, Guardian, and the language model, and the design of the risk
scoring system.

Implementation with Azure Services

We’ll leverage various Azure services to build the core functionalities of Guardian:

Azure AI Language: We’ll use this for NLP tasks, such as understanding the content and context of the interactions.

Azure Machine Learning: We’ll use this to train our risk assessment model. This model will be trained on a dataset of
policy-violating and non-violating interactions.
Azure Monitor and Azure Stream Analytics: These will be used for real-time monitoring of the interactions and the
performance of Guardian.
Azure AD Application Proxy: This will be used to implement the proxy server functionality in Guardian, allowing it to
intercept and analyze all interactions.

Database Setup

We’ll use Azure’s database management services to set up a robust and secure database. This database will log all
user interactions, risk scores, and actions taken by Guardian. It will be designed to handle high volumes of data and
provide fast query responses.

Integration and Testing

After implementing the individual components, we’ll integrate them and test the entire system. We’ll use Azure
DevOps for continuous integration and continuous delivery (CI/CD). This will allow us to quickly identify and fix any
issues, ensuring a smooth user experience.

Deployment

Once testing is complete, we’ll deploy Guardian on Azure. We’ll also set up the Azure Active Directory for secure
access to the admin dashboard. This will ensure that only authorized personnel can access the dashboard.

Monitoring and Updates

Post-deployment, we’ll use Azure Monitor to keep track of Guardian’s performance. This will provide us with valuable
insights, such as the most common types of policy violations and the effectiveness of the preventative measures.
Based on these insights, we’ll make necessary updates or improvements to Guardian.

User Feedback and Iteration

We’ll gather user feedback on the usability and effectiveness of Guardian. We’ll use this feedback to make iterative
improvements to the system, ensuring that it continues to effectively mitigate the risks associated with using large
language models.
Conclusion and Future Works

Conclusion

Guardian offers a robust solution to ensure responsible and ethical Generative AI use within organizations. By actively
identifying and preventing potential harm associated with AI, Guardian not only helps in compliance with
organizational policies and regulations but also fosters trust in AI adoption. With a focus on transparency, control, and
real-time monitoring, Guardian stands as a pivotal tool for building trust in AI through ethical practices and responsible
adoption.

Guardian’s integration with Azure services further enhances its capabilities, making it a scalable, reliable, and efficient
solution for AI risk management. The use of Azure AI Language for NLP, Azure Machine Learning for risk assessment, and
Azure Monitor for real-time monitoring, among others, ensures that Guardian is equipped with state-of-the-art
technology to effectively mitigate risks associated with large language models.

Future Works

Looking ahead, we envision several enhancements to Guardian:

1. Advanced NLP Techniques: We plan to incorporate more advanced NLP techniques for a deeper understanding of
the interactions. This could include sentiment analysis, named entity recognition, and more.
2. Improved Risk Assessment Model: We aim to continually improve the risk assessment model by training it on more
diverse and larger datasets. This would help in more accurate risk scoring.

3. User Feedback Integration: We plan to integrate a user feedback system to continuously improve Guardian based
on user experiences and suggestions.
4. Expansion to Other AI Models: While Guardian is currently designed for large language models, we aim to expand
its capabilities to monitor and manage risks associated with other types of AI models.

In conclusion, Guardian is not just a prototype but a step towards a future where AI is used responsibly and ethically. It
represents our commitment to ensuring that the benefits of AI are realized while minimizing the potential risks.