Dev Lead: @Brett Butterfield

Wiki: https://www.notion.so/cere/42-Hackathon-AI-Track-computer-vision-Wiki-1a6d800083d68088822af673ec278d2b

Description

This is a fun and challenging project that uses various Computer Vision (CV) related open source AI models to identify and label media in real-time for eventual search in a vector DB. Experience fine-tuning of LLMs and enabling a functional framework to build on your dream RAG or media search project.

Goal/Values

You will be able to quickly set up and interact with a number of photo and video streams that can be analyzed by open source models and datasets to see how media is by CV models, producing a summary of what the model predicts in the end.

Key value here is that you can experiment with how you can improve this process by tuning existing code/models or introducing new models that helps you to get better results!

Execution

• Clone the GitHub repo and follow the instructions in the Readme.md and Quick-Start guides https://github.com/brettb/cere-vision

  • Stephan Log
  • Pasha Log
  • Edgar Log
  • Nikolai Log
  • Stephan Log

• 🎯 Deliverables 2024-02-26 (Hackathon Day 1)

• 🎯 Deliverables 2024-02-27 (Hackathon Day 2)

• 🎯 Deliverables 2024-02-28 (Hackathon Day 3)

Quickstart guide

The project repos and its README.md file is the key “source of truth” that gives you the practical step by step guidance. A quick overview of it:

1. How to setup your environment and import a sample dataset
2. Run the base test in its default setting with the included code and model(s)
3. Analyze and visualize the result/scores
4. Your Challenge - Can you replace the included model with an alternate opensource model from Yolo11 and tune it to get a better results and additional classes for identification? See your photos, videos and a real-time camera feed identify objects in the scene. Determine the best balance in accuracy and performance for your system, from EDGE devices to high-end laptops there are no limits
5. 👨‍🍳 Start cooking on your execution!
   1. Log your work and findings in the execution log, which is synced above regularly in real-time. I have added a simple UI after the hackathon and have made some small enhancements. Feel free to reach out with questions and pull request if you find and fix a bug or add a feature
   2. Sync with the project lead so you can get constant guidance and also organize and track the group finding into shared tracking sheet

• The repos above is public, however if you have any issues with the access, ping @Brett Butterfield in the 42-schools-external on Slack channel or send a DM