So I decided to create a variant of the SparkOne Bot, using the same client side android application modifying it only slightly to include the necessary changes to switch from using a locally run transformer model to a locally run quantized Llama3 model.

This is a screenshot from the client side Application. It features text to speech and speech to text functionality, a quick response time and reasonably accurate answers as long as the queries are formed correctly. This is a current work in progress and some features are not fully refined as of yet.

I decided to keep the back end simple, using a Flask API endpoint for the Ollama model Llama3:latest. The endpoint also uses qdrant vectorized document corpus to allow for semantic searching.

Today I added a once per day script to scrape the organizational email account to gather information about all the departures and other news, including how much residents owed when leaving the oxford house they were a member or. This was accomplished using Selenium and some python and javascript.

So far, the email data is not as readily accessible by verbal prompt. I suppose it is because this is primarily NLP technology and the email data is scraped in typical email format. I will have to try some alterations to the method in the near future to see if this can be improved. I suspect data processing will be useful. Anyway, for a quick 1 day project I feel like it worked relatively well.

Summary of Latest Experiments:

Enhancing Oxford Bot: NLP Experiments and Outcomes

Today, we focused on improving Oxford Bot’s search capabilities, particularly for retrieving specific contact information. Here’s a summary of our experiments and outcomes:

1. Custom NodeParser Implementation: We created a ContactInfoParser to handle contact information separately from general documents. This improved the preservation of context for contact details.
2. Hybrid Retrieval System: We implemented a HybridContactRetriever that combines vector-based search with keyword matching. This approach aimed to enhance the retrieval of specific information like “treasurer of Cave Run”.
3. Chunking Strategies: We experimented with different chunk sizes and overlaps, ultimately settling on larger chunks (1024 tokens) with significant overlap (200 tokens) for non-contact documents. This improved the context preservation in general queries.
4. Entity Extraction: We integrated spaCy for named entity recognition, adding extracted entities to the context for more informed responses.
5. Fuzzy Matching: We implemented fuzzy search to catch near-matches and misspellings in user queries.
6. API Compatibility: We addressed issues related to llama_index API changes, ensuring our custom components work seamlessly with the latest version.

Outcomes:

• Improved retrieval of contact information, especially for specific queries.
• Enhanced general search capabilities due to better chunking strategies.
• More robust query handling with the addition of fuzzy matching and entity extraction.
• Resolved compatibility issues with the latest llama_index API.

While we made significant progress, fine-tuning the balance between specific (contact) and general queries remains an ongoing process. The experiments highlight the complexity of optimizing an AI assistant for diverse query types and the importance of continual refinement based on real-world usage.

Next steps include further testing with a wide range of queries and potential adjustments to the retrieval logic based on observed performance.

Oxford Bot(c) by www.sparkonelabs.com – Your home for AI and IT solutions.

Update – A diagram of the information processing as of api_flask_6-9.py:

TypeSense has been suggested as an improvement to the architecture of this project, the reference links are inlcuded below.

Oxford Bot Kotlin APIs and Package Versions

Gradle Plugin Versions

• Android Application Plugin: Not specified (using version catalog)
• Kotlin Android Plugin: Not specified (using version catalog)

Core Android and Kotlin Dependencies

• androidx.core:core-ktx: Not specified (using version catalog)
• androidx.lifecycle:lifecycle-runtime-ktx: Not specified (using version catalog)
• androidx.activity:activity-compose: Not specified (using version catalog)

Jetpack Compose

• androidx.compose.ui:ui: 1.6.6
• androidx.compose.ui:ui-graphics: Not specified (using version catalog)
• androidx.compose.ui:ui-tooling-preview: Not specified (using version catalog)
• androidx.compose.material:material: 1.6.6
• androidx.compose.runtime:runtime: 1.2.0
• androidx.compose.material:material-icons-core-android: 1.5.0

Testing Dependencies

• junit:junit: Not specified (using version catalog)
• androidx.test.ext:junit: Not specified (using version catalog)
• androidx.test.espresso:espresso-core: Not specified (using version catalog)
• androidx.compose.ui:ui-test-junit4: Not specified (using version catalog)

Debug Dependencies

• androidx.compose.ui:ui-tooling: Not specified (using version catalog)
• androidx.compose.ui:ui-test-manifest: Not specified (using version catalog)

Additional Dependencies

• androidx.appcompat:appcompat: 1.6.1
• org.jetbrains.kotlinx:kotlinx-coroutines-core: 1.6.0
• org.jetbrains.kotlinx:kotlinx-coroutines-android: 1.6.0
• com.squareup.retrofit2:retrofit: 2.9.0
• com.squareup.retrofit2:converter-gson: 2.9.0
• com.squareup.okhttp3:okhttp: 4.10.0

Custom APIs

• ApiService: Custom API service for communication with the backend
• TextToSpeech: Android’s built-in TextToSpeech API

Note: Some versions are managed through Gradle’s version catalog (libs.versions.toml) and are not directly specified in the build.gradle.kts file. You may need to check the version catalog file for the exact versions of these dependencies.

Another Day at the Lab:

Enhancing Oxford Bot’s Query Processing: A Day of Experimentation and Improvement

Today, we delved into refining the query processing capabilities of Oxford Bot, an AI assistant designed to provide information about Oxford House. Our focus was on improving the bot’s ability to understand and respond to user queries more accurately and efficiently. Here’s a summary of our experiments and findings:

1. Case-Insensitive Matching

We started by ensuring that Oxford Bot could match queries regardless of case. This was crucial for queries like “tradition two” to match headings like “TRADITION TWO” in the document corpus. We implemented a preprocess_text function to convert all text to lowercase during processing, ensuring case-insensitive matching.

2. Fuzzy Matching Refinement

Initial implementations of fuzzy matching led to an abundance of matches for common words like “the,” “of,” and “me.” To address this, we introduced a stop words list to exclude these common words from fuzzy matching. This significantly reduced noise in our search results and improved the relevance of matches.

3. Enhanced Fuzzy Search with Sliding Window Approach

We implemented an advanced fuzzy search method using a sliding window approach. This method starts with the full query and progressively removes words from both ends, searching for matches at each step. For example, a query like “what are the general principles of oxford house” would be searched as:

1. “general principles oxford house”
2. “general principles oxford” and “principles oxford house”
3. “general principles”, “principles oxford”, “oxford house”
4. Individual words: “general”, “principles”, “oxford”, “house”

This approach greatly improved our ability to find relevant matches, especially for longer queries or when relevant information is spread across multiple sentences.

4. Query Classification Improvement

We refined our is_simple_query function to use the enhanced fuzzy search method. This improved our ability to classify queries as simple (can be answered directly from the context) or complex (requiring more sophisticated processing).

Results and Observations

• The case-insensitive matching ensured that relevant information wasn’t missed due to capitalization differences.
• Implementing the stop words list in fuzzy matching significantly reduced false positives and improved the signal-to-noise ratio in our search results.
• The sliding window approach in fuzzy search enhanced our ability to handle a wider range of query formulations, making the bot more flexible and user-friendly.
• These improvements collectively led to more accurate query classification, allowing the bot to provide more relevant and concise responses for simple queries while appropriately engaging more complex processing for nuanced questions.

Conclusion

Today’s experiments have significantly enhanced Oxford Bot’s query processing capabilities. By implementing case-insensitive matching, refining fuzzy search, and introducing a sliding window approach, we’ve made the bot more robust and capable of handling a wider range of user queries. These improvements lay a solid foundation for further enhancements to Oxford Bot’s functionality and user experience.

Next steps include extensive testing with a variety of real-world queries, fine-tuning the fuzzy matching threshold, and potentially exploring more advanced natural language processing techniques to further improve query understanding and response generation.

References:

1. TypeSense, fast backend data retrieval – https://typesense.org/docs/overview/what-is-typesense.html
2. Linux Installation (TypeSense) – https://typesense.org/docs/guide/install-typesense.html#linux-binary