Vague/Complex Ask

In this example we ask a vague question that isn’t straightforward, and requires joining tables in a relational database (Chinook) to give a response. Let’s see how VerbaGPT does.

When I asked the question, I had “U2” in mind.

Video:

User question: Rank the top selling albums by that famous irish band.

VerbaGPT response:

Cost of query: $0.00

Answer:

Artist: U2, Album: Rattle And Hum, Total Sold: 17
Artist: U2, Album: Instant Karma: The Amnesty International Campaign to Save Darfur, Total Sold: 16
Artist: U2, Album: The Best Of 1980-1990, Total Sold: 11
Artist: U2, Album: War, Total Sold: 11
Artist: U2, Album: B-Sides 1980-1990, Total Sold: 11
Artist: U2, Album: How To Dismantle An Atomic Bomb, Total Sold: 10
Artist: U2, Album: Pop, Total Sold: 10
Artist: U2, Album: Zooropa, Total Sold: 9
Artist: U2, Album: Achtung Baby, Total Sold: 6
Artist: U2, Album: All That You Can't Leave Behind, Total Sold: 6

VerbaGPT answered the question correctly – (it read my mind!) Since VerbaGPT comes with an embedding framework, it did not have to be told where to look, only pointed to a SQL server instance that contains many different databases including one that has the Chinook database. Chinook is a standard SQL database used for testing. Further, VerbaGPT provided a complete response, along with a helpful option to save the datable to a csv file.

Here is a Python code snippet that should help you rank the top selling albums by that famous Irish band using the provided schema:

query = '''
SELECT TOP 10 Artist.Name, SUM(InvoiceLine.Quantity) AS TotalSold
FROM Artist
JOIN Album ON Artist.ArtistId = Album.ArtistId
JOIN Track ON Album.AlbumId = Track.AlbumId
JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId
GROUP BY Artist.Name
ORDER BY TotalSold DESC
'''

cursor.execute(query)
results = cursor.fetchall()

for row in results:
    print(row)

This code assumes that the famous Irish band you are referring to is U2. If it's a different band, you can replace `'U2'` in the `WHERE` clause with the name of the band you are interested in. Also, make sure to replace `cursor` with the name of your cursor object.

LangChain + OpenAI response:

Cost of query: $0.16

The langchain + OpenAI approach didn’t work for this query. For some reason, it wanted to fetch ArtistId=1, which isn’t what I wanted. And then in the answer it “hallucinated” the response that the AC/DC albums ‘For Those About to Rock We Salute You’ and ‘Let Thre Be Rock’ are from ‘that famous irish band’.

AC/DC are Australian from a quick search. Whether the band member’s roots are Irish, I don’t know 🙂

The langchain agent did need an explicit and direct connection to the relevant database (i.e., Chinook) in order to try to answer this question.