Sandbox

The inspect_viz.sandbox module includes prototypes of various higher level visualization functions. We’d like to see lots of contributions here to help develop and refine a set of functions that will be promoted into the main package.

Demos

The following demonstrate using various inspect_viz.sandbox functions:

  • evals_summary—Demonstrates evals_summary_plot() and evals_summary_table(), which produce summaries of scores by model and task name for a set of log files.

Contributing

To contribute functions to inspect_viz.sandbox functions:

  1. Add your code to src/inspect_viz/sandbox, being sure to export your functions and other types from the __init__.py file within the module.

  2. Add a demo notebook to the docs/sandbox directory which demonstrates how to use your sandbox funcdtions (your demo notebook should use the logs in docs/sandbox/logs).

  3. Submit a PR with your changes.

There are some additional steps required to integrate your functions with the website/documentation—we’ll take care of those when we merge the PR.

Important

The logs in docs/sandbox/logs are checked into the repo using Git LFS. Ensure that you have installed Git LFS (git lfs --version) and then do a git lfs pull to ensure you have local copies of the logs.