Part 3: Summarize Results (AWS Bedrock with Meta’s Llama 3.2)

Finally, we deploy a Pulsar Function that uses Meta’s Llama 3.2 to summarize the data returned by the similarity search.

Deploy BedrockSummary1

Code for this example can be found in streamnativerag2 class BedrockSummarize.

To deploy:

1. Navigate terminal to the folder containing streamnativerag2.zip.
2. Execute the following pulsarctl command. Be sure to edit the tenant in three places in the command where you would like to deploy the Pulsar Function (–input, –output, –tenant).

pulsarctl functions create --classname streamnativerag2.BedrockSummarize --py ./streamnativerag2.zip --inputs summitstudent1/developer/simout1 --output summitstudent1/developer/summ1 --tenant summitstudent1 --namespace developer --name BedrockSummary1 --secrets '{"BEDROCKSECRET1": {"path": "bedrocksecret", "key": "accesskey"}, "BEDROCKSECRET2": {"path": "bedrocksecret", "key": "secretaccesskey"}}'

If the Pulsar Function starts deploying, you should see:

Created BedrockSummary1 successfully

It may take a minute or two for the function to deploy. Once fully deployed, you should see BedrockSummary1 has a Status of Running. If you see any System Exceptions, view Troubleshooting Pulsar Functions.

The Python Function will create a query with “summarize the following data into a single sentence without adding any additional information: “, followed by the two results returned by the similarity search.

Test BedrockSummary1

To test BedrockSummary1, we can publish a string message to input2.

curl -X POST https://<SERVER ENDPOINT>/admin/rest/topics/v1/persistent/summitstudent1/developer/input2/message \
  --header 'Authorization: Bearer <JWT TOKEN>' \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/octet-stream' \
  --data-binary 'I like cats.'

If BedrockSummary1 triggered, you should see 1 in the Messages column.

If you see any System Exceptions or User Exceptions, view Troubleshooting Pulsar Functions.

We will use the UI to check the results in the output topic. Navigate to the correct tenant and namespace where you deployed the Pulsar Function. Select the summ1 topic. After creating a subscription, we can peek at the message.

You can see that the two results of “Cats are friendly animals.” and “Cats are colorful.” were summarized by Meta’s Llama 3.2 to “Cats are known for their distinctive colors and friendly nature.”

Deleting Pulsar Functions

To delete the Pulsar Functions, execute the following from pulsarctl.

pulsarctl functions delete --tenant summitstudent1 --namespace developer --name Embed1
pulsarctl functions delete --tenant summitstudent1 --namespace developer --name Embed2
pulsarctl functions delete --tenant summitstudent1 --namespace developer --name ZillizSearch1
pulsarctl functions delete --tenant summitstudent1 --namespace developer --name BedrockSummary1
Also delete the Milvus Connector using StreamNative UI.