# Users
Welcome to the user documentation for Jupyter AI.
If you are interested in contributing to Jupyter AI,
please see our {doc}`contributor's guide `.
If you would like to build applications that enhance Jupyter AI,
please see the {doc}`developer's guide `.
## Prerequisites
You can run Jupyter AI on any system that can run a supported Python version
from 3.8 to 3.11, including recent Windows, macOS, and Linux versions.
If you use `conda`, you can install Python 3.11 in your environment by running:
```
conda install python=3.11
```
The `jupyter_ai` package, which provides the lab extension and user interface in
JupyterLab, depends on JupyterLab 4. If upgrading to JupyterLab 4 is not
possible in your environment, you should install `jupyter_ai` v1.x instead.
See "Installation" for more details.
You can install JupyterLab using `pip` or `conda`.
1. via `pip`:
```
# change 4.0 to 3.0 if you need JupyterLab 3
pip install jupyterlab~=4.0
```
2. via `conda`:
```
# change 4.0 to 3.0 if you need JupyterLab 3
conda config --add channels conda-forge
conda config --set channel_priority strict
conda install jupyterlab~=4.0
```
The `jupyter_ai_magics` package, which provides exclusively the IPython magics,
does not depend on JupyterLab or `jupyter_ai`. You can install
`jupyter_ai_magics` without installing `jupyterlab` or `jupyter_ai`.
If you have both `jupyter_ai_magics` and `jupyter_ai` installed, you should
have the same version of each, to avoid errors.
Jupyter AI internally uses Pydantic v1 and should work with either Pydantic
version 1 or version 2. For compatibility, developers using Pydantic V2
should import classes using the `pydantic.v1` package. See the
[LangChain Pydantic migration plan](https://python.langchain.com/docs/guides/pydantic_compatibility)
for advice about how developers should use `v1` to avoid mixing v1 and v2
classes in their code.
## Installation
### Installation via `pip`
To install the JupyterLab extension, you can run:
```
pip install jupyter-ai
```
The latest major version of `jupyter-ai`, v2, only supports JupyterLab 4. If you
need support for JupyterLab 3, you should install `jupyter-ai` v1 instead:
```
pip install jupyter-ai~=1.0
```
If you are not using JupyterLab and you only want to install the Jupyter AI `%%ai` magic, you can run:
```
$ pip install jupyter-ai-magics
```
`jupyter-ai` depends on `jupyter-ai-magics`, so installing `jupyter-ai`
automatically installs `jupyter-ai-magics`.
### Installation via `pip` or `conda` in a Conda environment (recommended)
We highly recommend installing both JupyterLab and Jupyter AI within an isolated
Conda environment to avoid clobbering Python packages in your existing Python
environment.
First, install
[conda](https://conda.io/projects/conda/en/latest/user-guide/install/index.html)
and create an environment that uses Python 3.11:
$ conda create -n jupyter-ai python=3.11
$ conda activate jupyter-ai
Then, use `conda` to install JupyterLab and Jupyter AI in this Conda environment.
$ conda install -c conda-forge jupyter-ai # or,
$ conda install conda-forge::jupyter-ai
When starting JupyterLab with Jupyter AI, make sure to activate the Conda
environment first:
```
conda activate jupyter-ai
jupyter lab
```
## Uninstallation
If you installed Jupyter AI using `pip`, to remove the extension, run:
$ pip uninstall jupyter-ai
or
$ pip uninstall jupyter-ai-magics
If you installed Jupyter AI using `conda`, you can remove it by running:
$ conda remove jupyter-ai
or
$ conda remove jupyter-ai-magics
## Model providers
Jupyter AI supports a wide range of model providers and models. To use Jupyter AI with a particular provider, you must install its Python packages and set its API key (or other credentials) in your environment or in the chat interface.
Jupyter AI supports the following model providers:
| Provider | Provider ID | Environment variable(s) | Python package(s) |
|---------------------|----------------------|----------------------------|---------------------------------|
| AI21 | `ai21` | `AI21_API_KEY` | `ai21` |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | `langchain-anthropic` |
| Anthropic (chat) | `anthropic-chat` | `ANTHROPIC_API_KEY` | `langchain-anthropic` |
| Bedrock | `bedrock` | N/A | `boto3` |
| Bedrock (chat) | `bedrock-chat` | N/A | `boto3` |
| Cohere | `cohere` | `COHERE_API_KEY` | `cohere` |
| ERNIE-Bot | `qianfan` | `QIANFAN_AK`, `QIANFAN_SK` | `qianfan` |
| Gemini | `gemini` | `GOOGLE_API_KEY` | `langchain-google-genai` |
| GPT4All | `gpt4all` | N/A | `gpt4all` |
| Hugging Face Hub | `huggingface_hub` | `HUGGINGFACEHUB_API_TOKEN` | `huggingface_hub`, `ipywidgets`, `pillow` |
| NVIDIA | `nvidia-chat` | `NVIDIA_API_KEY` | `langchain_nvidia_ai_endpoints` |
| OpenAI | `openai` | `OPENAI_API_KEY` | `langchain-openai` |
| OpenAI (chat) | `openai-chat` | `OPENAI_API_KEY` | `langchain-openai` |
| SageMaker | `sagemaker-endpoint` | N/A | `boto3` |
The environment variable names shown above are also the names of the settings keys used when setting up the chat interface.
If multiple variables are listed for a provider, **all** must be specified.
To use the Bedrock models, you need access to the Bedrock service. For more information, see the
[Amazon Bedrock Homepage](https://aws.amazon.com/bedrock/).
To use Bedrock models, you will need to authenticate via
[boto3](https://github.com/boto/boto3).
You need the `pillow` Python package to use Hugging Face Hub's text-to-image models.
You can find a list of Hugging Face's models at [https://huggingface.co/models](https://huggingface.co/models).
To use NVIDIA models, create a free account with the [NVIDIA NGC service](https://catalog.ngc.nvidia.com/), which hosts AI solution catalogs, containers, models, and more. Navigate to Catalog > [AI Foundation Models](https://catalog.ngc.nvidia.com/ai-foundation-models), and select a model with an API endpoint. Click "API" on the model's detail page, and click "Generate Key". Save this key, and set it as the environment variable `NVIDIA_API_KEY` to access any of the model endpoints.
SageMaker endpoint names are created when you deploy a model. For more information, see
["Create your endpoint and deploy your model"](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-deployment.html)
in the SageMaker documentation.
To use SageMaker's models, you will need to authenticate via
[boto3](https://github.com/boto/boto3).
For example, to use OpenAI models, use the chat interface settings panel to choose the OpenAI language model:
Then, enter your API key in the 'API Keys' section.
:::{attention}
:name: open-ai-cost
Model providers may charge users for API usage. Jupyter AI users are
responsible for all charges they incur when they make API requests. Review your model
provider's pricing information before submitting requests via Jupyter AI.
:::
## The chat interface
The easiest way to get started with Jupyter AI is to use the chat interface.
:::{attention}
:name: open-ai-privacy-cost
The chat interface sends data to generative AI models hosted by third parties. Please review your model provider's privacy policy to understand how it may use the data you send to it. Review its pricing model so that you understand your payment obligations when using the chat interface.
:::
Once you have started JupyterLab, click the new "chat" icon in the left side panel to open the chat interface. You can right-click on the panel icon and move it to the other side, if you prefer.
The first time you open the chat interface, Jupyter AI will ask you which models you want to use as a language model and as an embedding model. Once you have made your selections, the UI may display text boxes for one or more settings keys.
:::{admonition} Language models and embedding models
:class: tip
:name: language-models-and-embedding-models
Users may select a language model and, optionally, an embedding model. You should select one of each so that you can use the full functionality of the chat interface.
A **language model** responds to users' messages in the chat panel. It accepts a prompt and produces a response. Language models are typically *pre-trained*; they are ready to use, but their training sets are biased and incomplete, and users need to be aware of their biases when they use the chat interface.
An **embedding model** is used when [learning and asking about local data](#learning-about-local-data). These models can transform your data, including documents and source code files, into vectors that can help Jupyter AI compose prompts to language models.
Your language model and your embedding model do not need to be provided by the same vendor, but you will need authentication credentials for each model provider that you use.
:::
Before you can use the chat interface, you need to provide your API keys for the model providers that you have selected. Paste or type your keys into the boxes provided.
Once you have set all the necessary keys, click the "back" (left arrow) button in the upper-left corner of the Jupyter AI side panel. The chat interface now appears, and you can ask a question using the message box at the bottom.
To compose a message, type it in the text box at the bottom of the chat interface and press ENTER to send it. You can press SHIFT+ENTER to add a new line. (These are the default keybindings; you can change them in the chat settings pane.) Once you have sent a message, you should see a response from Jupyternaut, the Jupyter AI chatbot.
The chat backend remembers the last two exchanges in your conversation and passes them to the language model. You can ask follow up questions without repeating information from your previous conversations. Here is an example of a chat conversation with a follow up question:
#### Initial question
#### Follow-up question
### SageMaker endpoints usage
Jupyter AI supports language models hosted on SageMaker endpoints that use JSON
schemas. The first step is to authenticate with AWS via the `boto3` SDK and have
the credentials stored in the `default` profile. Guidance on how to do this can
be found in the
[`boto3` documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html).
When selecting the SageMaker provider in the settings panel, you will
see the following interface:
Each of the additional fields under "Language model" is required. These fields
should contain the following data:
- **Endpoint name**: The name of your endpoint. This can be retrieved from the
AWS Console at the URL
`https://.console.aws.amazon.com/sagemaker/home?region=#/endpoints`.
- **Region name**: The AWS region your SageMaker endpoint is hosted in, such as `us-west-2`.
- **Request schema**: The JSON object the endpoint expects, with the prompt
being substituted into any value that matches the string literal `""`.
In this example, the request schema `{"text_inputs":""}` generates a JSON
object with the prompt stored under the `text_inputs` key.
- **Response path**: A [JSONPath](https://goessner.net/articles/JsonPath/index.html)
string that retrieves the language model's output from the endpoint's JSON
response. In this example, the endpoint returns an object with the schema
`{"generated_texts":["