Nvidia is introducing a new AI chatbot called Chat with RTX, which is designed to run locally on users’ PCs. This demo app allows individuals to input YouTube videos and personal documents, enabling the chatbot to generate summaries and provide relevant answers based on the data. The app requires an RTX 30- or 40-series GPU with at least 8GB of VRAM to operate. Although the app is still in its early stages and has some rough edges, it shows promise for data research purposes, particularly for journalists and those who need to analyze collections of documents.
Chat with RTX can handle YouTube videos by allowing users to input a video URL and search for specific mentions or generate summaries of the entire video. This feature is especially useful for searching through video podcasts and locating specific references or mentions. However, the app is not flawless and may encounter some bugs. For example, it may download the transcript of a different video instead of the one that was queried.
Despite these occasional issues, Chat with RTX has proven to be valuable for tasks such as analyzing court documents, scanning through PDFs, and fact-checking data. Microsoft’s Copilot system, for example, struggles with handling PDFs within Word, whereas Chat with RTX excels at extracting key information from PDF files. Additionally, the app provides near-instant responses with minimal lag, unlike cloud-based chatbots such as ChatGPT or Copilot.
However, it’s important to note that Chat with RTX is still in the early stages and feels more like a developer demo. It requires the installation of a web server and Python instance on users’ PCs and utilizes Mistral or Llama 2 models to process the data provided. Users with an RTX GPU can benefit from the accelerated query speed achieved using Nvidia’s Tensor cores. The installation process can take approximately 30 minutes, and the app itself is quite large in size (approximately 40GB). While certain limitations and known issues exist, such as inaccurate source attribution, the app offers a glimpse into the potential of AI chatbots that run locally on personal computers.
In conclusion, Nvidia’s Chat with RTX showcases the future possibilities of AI chatbots operating on users’ own PCs. While it is currently in its early stages and comes with a few limitations, it demonstrates the value of local AI tools for data research, document analysis, and fact-checking. As development progresses, Chat with RTX has the potential to become an essential tool for journalists, researchers, and anyone in need of efficient data analysis.
