OpenAI : which azure OpenAI function need to use for rule checking

Question

I have a query regarding utilizing OpenAI. I have large documents containing rules and I want to check if my content violates any of these rules. Additionally, I need to identify which specific rules are being violated. Could someone guide me on which Azure OpenAI function I should use for this purpose?

Answer 1

Hello Su Myat Hlaing,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

Problem

Sequel to your questions, I understand that you require assistance in leveraging Azure OpenAI services to perform rule checking on extensive documents. The goal is to detect violations of predefined rules within the content and precisely identify which rules are being violated.

Scenarios

Based on the information provided, I created a unique scenario for your use case and to provide more insight. You work on a project for a regulatory compliance department tasked with ensuring that documents adhere to a set of predefined rules and regulations. Your organization deals with large volumes of documents, and manually checking each one for compliance is time-consuming and error-prone. To streamline this process, you decide to leverage Azure OpenAI services.

The questions arise:

Rule Checking with Azure OpenAI: What Azure OpenAI function or service should be used for rule checking on large documents?

• How can violations of rules within the content be detected using Azure OpenAI?
• How can we accurately identify which specific rules are being violated within the documents, not just the presence of violations?

Solution

To effectively address the scenario and questions, there are options for you in Azure OpenAI.

Firstly, Azure AI Search:

You can implement knowledge mining with Azure AI Search for rule checking purposes according to a suggest by AI. Azure AI Search offers various features and capabilities that can be leveraged for this task. Here's how you can use Azure AI Search for rule checking:

• Begin by creating an index in Azure AI Search to store your documents. Define the fields in your index that will be used for rule checking, such as document content, metadata, or any other relevant information.
• Create a skillset in Azure AI Search to define the processing pipeline for your documents. Within the skillset, you can incorporate custom skills or built-in skills provided by Azure AI Search to extract and analyze information from the documents.
• Use custom skills within the skillset to extract rules from the documents. This can involve techniques like keyword extraction, pattern matching, or regular expressions to identify and extract rule-related information from the text.
• Once the rules are extracted, implement logic within your application to evaluate the documents against the extracted rules. This can be done by comparing the content of the documents with the extracted rules and identifying any violations.
• Finally, use Azure AI Search's querying and visualization capabilities to generate reports and visualize the results of the rule checking process. This can help stakeholders easily understand the compliance status of the documents and any violations detected.

Other Solution

In addition to the above, other solutions within Azure AI ecosystem that can be considered for rule checking include:

• Azure Cognitive Search: This service offers powerful indexing, searching, and analyzing capabilities for structured and unstructured data. It can be customized to extract and analyze information from documents for rule checking purposes.
• Azure Machine Learning: You can leverage Azure Machine Learning to build custom machine learning models for rule checking. This approach allows for more sophisticated analysis of documents and detection of rule violations based on learned patterns and features.
• Azure Cognitive Services (Language Understanding): Services like Language Understanding (LUIS) within Azure Cognitive Services can be used to train models to understand and interpret natural language input. This can be useful for identifying rule violations expressed in different forms or variations within the documents.

By exploring these alternative solutions within the Azure AI ecosystem, you can tailor your approach to rule checking based on your specific requirements and use cases.

Finally

Best Solution

Based on the scenario, and the questions, my best suggested solution for performing rule checking on large documents while accurately identifying specific rule violations is to use Azure Text Analytics with Custom Named Entity Recognition (NER) feature.

Azure Text Analytics

This solution offers the following advantages:

• Customization: With Custom NER, you can define and train models tailored to your specific rules, allowing for precise detection of rule violations within the documents.
• Scalability: Azure Text Analytics provides scalable processing capabilities, enabling efficient analysis of large volumes of documents for rule checking purposes.
• Integration: Azure Text Analytics seamlessly integrates with other Azure services, facilitating easy implementation and integration into existing workflows and applications.
• Accuracy: The rule-based Named Entity Recognition provided by Text Analytics for Health feature ensures accurate identification of rule-related entities and violations within the documents.

While other Azure AI services offer valuable features for text analysis and processing, the Custom NER functionality within Azure Text Analytics is specifically designed for tasks like rule checking, making it the most suitable solution for the given scenario.

Things you will need to do to achieve best solution.

• Choose Azure Text Analytics Service by select the appropriate Azure Text Analytics service for rule checking. Given the requirement for identifying specific rules, we'll focus on the Custom Named Entity Recognition (NER) feature.
• Create Custom NER Models to develop custom NER models representing the rules to be checked against. Each rule should be defined as an entity within the model.
• Analyze Documents by utilize the custom NER models to analyze the documents. This involves extracting entities from the text and identifying instances where rules are mentioned or violated.
• Identify Violations after analyzing the documents, identify instances where the extracted entities indicate violations of the predefined rules.
• Map Violations to Rules by map the identified violations back to the corresponding rules. This step involves associating each detected violation with the specific rule it pertains to.

I can provide you some python code snippets to achieve above with some step-by-step instructions if this is your choice.

References

You can read more from the right side of this page.

Accept Answer

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

Please remember to "Accept Answer" if answer helped, so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam

Answer 2

@Su Myat Hlaing Please go through this page on content filtering with Azure OpenAI. AFAIK the set of rules or an API is not exposed to pre-run your documents, based on the prompts and completions the filtering kicks into effect if they fall in any of the categories mentioned.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.