December 29, 2023
Automated Testing with Query Vary
Learn how to use the Scheduler feature to automate your prompt testing process

When it comes to Large Language Models, their ever changing nature means that the responses you get can change as well. A prompt response that you received in March might not be the same in September. Therefore, it is essential to do continual testing of your prompts in order to make sure it is consistently providing reliable and accurate output.

To address this issue, we've added a new feature to help you schedule your prompt evaluation called Scheduler.

The Scheduler functionality enables automated execution of tests that were created in the Test Lab. It offers flexibility in scheduling test runs at preferred intervals, including options for minute, hourly, daily, weekly, monthly, or yearly frequencies.

You can follow these simple steps to run automated tests:

  1. Run a test in the Test Lab then navigate to the Scheduler tab.
  2. Click on the Create Event button

Query Vary - Scheduler Screen

  1. Enter your desired Test Name then select the test that you would like to automate the testing of.
  2. Select your desired interval of testing ( by minute, hour, daily, weekly or monthly) then click Create

Query Vary - Create Test Screen

Your scheduled test will now run at your desired interval. An email notification will be sent to you after each test run, keeping you informed of the test activity. You can also Pause and Update the scheduled test at any moment by clicking on the test name and navigating to the bottom of the screen. To Delete the scheduled test, you can click on the Bin icon that is located in the Actions column.

Query Vary - Scheduler with added test screen

Happy prompting everyone!

To try out Query Vary click:

You can also join our discord :

Try out Query Vary

Fast-track your LLM development with Query Vary. Be a part of prompt excellence and witness the shift in prompt design, testing, and refinement!

Book a Demo

Recent Blogs posted from our team

Check out our other blog posts to help you learn more about our product and also Large Language Models