Vicuna Chatbot
INTRODUCTION
According to lmsys.org; "Vicuna-13B is an open-source chatbot trained by fine-tuning LLaMA on user-shared
conversations collected from ShareGPT. Preliminary evaluation using GPT-4 as a
judge shows Vicuna-13B achieves more than 90% quality of OpenAI ChatGPT and Google
Bard while outperforming other models like LLaMA and Stanford Alpaca in more than 90% of cases.
The cost of training Vicuna-13B was around $300."
It is an auto-regressive language model, based on the transformer architecture.
Creation of Vicuna was a joint effort with collaborators from multiple institutions, including UC Berkeley, CMU, Stanford, UC San Diego, and MBZUAI. The blog entry outlining how Vicuna was created is located here *click here*.
Below are textual extracts from the lmsys.org blog entry. Reading the original blog entry is a must for understanding how this chatbot was implemented.
Inspired by the Meta LLaMA and Stanford Alpaca project, Vicuna-13B is an open-source chatbot backed by an enhanced dataset. By fine-tuning a LLaMA base model on user-shared conversations collected from ShareGPT.com, Vicuna-13B has demonstrated competitive performance compared to other open-source models (Stanford Alpaca).
Vicuna was created by fine-tuning a LLaMA base model using approximately 70,000 user-shared conversations gathered from ShareGPT.com through public APIs. Vicuna was trained between March 2023 and April 2023. Similar to other large language models, Vicuna has certain limitations. For instance, it is not good at tasks involving reasoning or mathematics, and it may have limitations in accurately identifying itself or ensuring the factual accuracy of its outputs. Additionally, it has not been sufficiently optimized to guarantee safety or mitigate potential toxicity or bias.
A preliminary evaluation of the model quality was conducted by creating a set of 80 diverse questions and utilizing GPT-4 to judge the model outputs.
As an experimenter, one of its advantages is that it can be downloaded and run on an average PC with sufficient memory. It requires at least 10GB of memory but runs best on faster systems with more memory.
The program leverages the vector processing (AVX) capabilities of modern Intel and AMD CPUs.
INSTALLATION
A github PowerShell script is available for Windows installation that automates
the full installation.
The script automates the process of setting up and running VICUNA on a CPU (
without a graphics card) using the llama.cpp library and a pre-trained GGML Vicuna model.
The script downloads and extracts the required files, creates a batch file to run VICUNA,
and creates a desktop shortcut to launch the batch file.
*click here* for the instructions.
Installation is also summarized below:
For the automatic installation (suggested), follow the steps below:
- In the Windows searchbar, lower left side of screen, type powershell in the search box and hit enter.
- Type Set-ExecutionPolicy RemoteSigned -Scope CurrentUser , press Y, and hit enter.
- Type irm bit.ly/vicuna_ps1 | iex and hit enter.
Unpacking the distribution
Select the AVX setting available on your desktop CPU. Core i3/i5/i7/i9 support AVX, AVX-2 and AVX-512, whereas Pentium and Celeron CPUs do not. AVX (Advanced Vector Extensions) is a set of instructions for Intel and AMD CPU's that use x86 architecture. These instructions allow more efficient handling of large data sets and improved performance in certain types of computations. The benefits of AVX include improved performance in scientific simulations, machine learning, and other high-performance computing applications. Additionally, AVX can enable better performance in data compression and encryption tasks.
Select AVX
More information on the AVX vector instructions is availale on Wikipedia *click here* .
An Intel i5 4670 quad core CPU running at 3.4GHz with 16GB of DDR3 memory and striped disks was used for this Vicuna installation. The package is installed in the C:\VICUNA directory.
Intel i5 CPU
Installation complete
When running the Vicuna batch script, by clicking on the desktop icon, this error came up. This is because the C++ redistributable library is missing. It must be installed on your computer before Vicuna will run. The file name installed was: VC_redist.X64, you can search the web for recent copy (Microsoft Visual C++ 2015-2022 redistributable x64).
Problems with runtime
By clicking on the Vicuna deskop icon, the startup batch script is initiated. This may take a bit of time to complete, depending on the speed of your computer, so be patient before entering instructions.
To test Vicuna's functionality, a few questions were posed. Note that it prompts you with "### Human:" to enter your question.
Running Vicuna
Interacting with chat
A coding query
Longer queries can be submitted over multiple lines. A backslash must be added at the end of each line to denote the question occurs over multiple lines.
Submitting a multi-line query
A quick note on some of the responses provided above. Neither the DCL script nor the generated FORTRAN program code actually worked on VMS. You could say Vicuna "hallucinated" in responding to the coding requests. I had the same issue with ChatGPT generated FORT77 code. Yet, I've read positive feedback that ChatGPT provides great coding tips.
Adding Numbers - FORT77 program compilation result
My explanation for these hallucinations is that very few people actually code in FORT77 these days. As well, the VMS operating system, although making a comeback, is today a niche operating system that no longer has a large installed user base as it enjoyed in the 1980s. These factors lend to a lack of knowledge on such languages as FORT77, APL or DCL.
Testing Vicuna versus ChatGPT 3.5 Responses
As a test of Vicuna's capabilities, I submitted a request to both ChatGPT and Vicuna: "Create a 3 day vacation itinerary for Seattle, Washington. Assume I have a rental car. Suggest accommodation options."
ChatGPT Response (1 minute to generate):
Day 1: Morning: Start your day with a visit to Pike Place Market, one of Seattle's most iconic attractions. Explore the market, enjoy the fresh produce, and grab a delicious breakfast at one of the local eateries. Afterward, drive to the Space Needle, Seattle's most recognizable landmark. Take the elevator to the top for panoramic views of the city and its surroundings. Next, head to the Chihuly Garden and Glass, located near the Space Needle. Admire the breathtaking glass artwork created by renowned artist Dale Chihuly. Afternoon: Drive to the Ballard neighborhood and have lunch at one of the trendy restaurants or food trucks in the area. Visit the Ballard Locks, a unique and operational set of locks that connect Puget Sound with the Lake Washington Ship Canal. Watch boats pass through and visit the fish ladder to see salmon migrating. Evening: Drive to Kerry Park in the Queen Anne neighborhood for a stunning view of the Seattle skyline, especially during sunset. Enjoy dinner at one of the restaurants in the nearby Queen Anne area. Accommodation suggestion for Day 1: Consider staying in the downtown area or the nearby neighborhoods such as Belltown or South Lake Union. Some options include The Westin Seattle, The Edgewater, or the Hyatt Regency Seattle. Day 2: Morning: Take a scenic drive to Olympic National Park, approximately a two-hour drive from Seattle. Explore the park's diverse landscapes, including mountains, rainforests, and stunning coastlines. You can hike one of the trails, visit the Hoh Rainforest, or explore the beaches. Afternoon: Visit the nearby city of Port Townsend, known for its Victorian architecture and charming downtown area. Take a stroll through the historic district, browse local shops, and have lunch at one of the waterfront restaurants. Evening: Return to Seattle and explore the vibrant Capitol Hill neighborhood. Enjoy dinner at one of the many eclectic restaurants or try out some of the popular bars and nightlife spots in the area. Accommodation suggestion for Day 2: Stay in the Capitol Hill neighborhood for convenient access to the vibrant nightlife. Consider options such as The Sound Hotel Seattle Belltown or The Charter Hotel Seattle. Day 3: Morning: Visit the Museum of Pop Culture (MoPOP) located near the Space Needle. Explore the exhibits dedicated to music, pop culture, and science fiction. Afterward, drive to the nearby Seattle Center and visit the Pacific Science Center, where you can engage in interactive exhibits and hands-on activities. Afternoon: Drive to the Fremont neighborhood and have lunch at one of the local cafes or restaurants. Take a walk along the Fremont Troll, a whimsical sculpture located under the Aurora Bridge. Visit the Woodland Park Zoo, home to a wide variety of animals and beautifully landscaped grounds. Evening: Drive to the waterfront area and take a leisurely stroll along the Seattle waterfront. Enjoy the scenic views of Puget Sound and the Olympic Mountains. End your trip with a delicious seafood dinner at one of the waterfront restaurants. Accommodation suggestion for Day 3: Consider staying in the waterfront area or the nearby Belltown neighborhood. Options include The Inn at the Market or Thompson Seattle. Note: It's advisable to check the opening hours and availability of attractions and activities in advance, as they may be subject to change.Vicuna Response (10 minutes to generate on desktop):
Vicuna response
Overall Vicuna did a good job creating an itinerary in comparison to ChatGPT. I actually like the Vicuna's responses best. Vicuna suggested five hotels to stay at but the screen capture cut off the last two.
Given the i5 hardware it was hosted on (no GPU), Vicuna took 10 minutes to generate the answer. Regardless, it is still amazing that this knowledge base is available on a modest desktop computer. Using a GPU would certianly have accelerated the output, but what is more important to me is the quality of the response (excluding FORT77 code). The Vicuna creators mention that it is 90% comparable to ChatGPT, the comparisons for this example indicate it to be true.
I can see that in time this data becomes stale and the language model will need to be refreshed. For example, the recommendation to stay at the Westin Seattle may no longer apply five years from now as the hotel could be bought by Hilton or shut down altogether.
SOURCES
- *click here* lmsys.org - Vicuna home page.
- *click here* Github.com - Automatic installation script for Vicuna (CPU-only).
- *click here* hub.tcno.co - This installation guide is for the GPU version of Vicuna and Oobabooga. Worth a look but assumes Vicuna is installed on Unix/Linux.
- *click here* HuggingFace - vicuna-13b-delta-v1.1 model.
- *click here* tc.ht - an open-source Windows PowerShell script that downloads Oobabooga and Vicuna (7B and/or 13B, GPU and/or CPU), as well as automatically sets up a Conda or Python environment, and creates a desktop shortcut. Run iex (irm vicuna.tc.ht) in PowerShell, and a new oobabooga-windows folder will appear, with everything set up. The script takes you through all the steps as it goes. The one-line auto-installer for Ooba itself is just iex (irm ooba.tc.ht). This uses the default model downloader, and launches it as normal.
Compiled on 12-18-2023 21:38:52