assistants/deep-dive.md +89 −89
23 23
24For example, to create an Assistant that can create data visualization based on a `.csv` file, first upload a file.24For example, to create an Assistant that can create data visualization based on a `.csv` file, first upload a file.
25 25
26```python
27file = client.files.create(
28 file=open("revenue-forecast.csv", "rb"), purpose="assistants"
29)
30```
31
32```javascript26```javascript
33const file = await openai.files.create({27const file = await openai.files.create({
34 file: fs.createReadStream("revenue-forecast.csv"),28 file: fs.createReadStream("revenue-forecast.csv"),
36});30});
37```31```
38 32
33```python
34file = client.files.create(
35 file=open("revenue-forecast.csv", "rb"), purpose="assistants"
36)
37```
38
39```bash39```bash
40curl https://api.openai.com/v1/files \40curl https://api.openai.com/v1/files \
41 -H "Authorization: Bearer $OPENAI_API_KEY" \41 -H "Authorization: Bearer $OPENAI_API_KEY" \
46 46
47Then, create the Assistant with the `code_interpreter` tool enabled and provide the file as a resource to the tool.47Then, create the Assistant with the `code_interpreter` tool enabled and provide the file as a resource to the tool.
48 48
49```python
50assistant = client.beta.assistants.create(
51 name="Data visualizer",
52 description="You are great at creating beautiful data visualizations. You analyze data present in .csv files, understand trends, and come up with data visualizations relevant to those trends. You also share a brief text summary of the trends observed.",
53 model="gpt-4o",
54 tools=[{"type": "code_interpreter"}],
55 tool_resources={"code_interpreter": {"file_ids": [file.id]}},
56)
57```
58
59```javascript49```javascript
60const assistant = await openai.beta.assistants.create({50const assistant = await openai.beta.assistants.create({
61 name: "Data visualizer",51 name: "Data visualizer",
71});61});
72```62```
73 63
64```python
65assistant = client.beta.assistants.create(
66 name="Data visualizer",
67 description="You are great at creating beautiful data visualizations. You analyze data present in .csv files, understand trends, and come up with data visualizations relevant to those trends. You also share a brief text summary of the trends observed.",
68 model="gpt-4o",
69 tools=[{"type": "code_interpreter"}],
70 tool_resources={"code_interpreter": {"file_ids": [file.id]}},
71)
72```
73
74```bash74```bash
75curl https://api.openai.com/v1/assistants \75curl https://api.openai.com/v1/assistants \
76 -H "Authorization: Bearer $OPENAI_API_KEY" \76 -H "Authorization: Bearer $OPENAI_API_KEY" \
100 100
101You can create a Thread with an initial list of Messages like this:101You can create a Thread with an initial list of Messages like this:
102 102
103```python
104thread = client.beta.threads.create(
105 messages=[
106 {
107 "role": "user",
108 "content": "Create 3 data visualizations based on the trends in this file.",
109 "attachments": [
110 {"file_id": file.id, "tools": [{"type": "code_interpreter"}]}
111 ],
112 }
113 ]
114)
115```
116
117```javascript103```javascript
118const thread = await openai.beta.threads.create({104const thread = await openai.beta.threads.create({
119 messages: [105 messages: [
131});117});
132```118```
133 119
120```python
121thread = client.beta.threads.create(
122 messages=[
123 {
124 "role": "user",
125 "content": "Create 3 data visualizations based on the trends in this file.",
126 "attachments": [
127 {"file_id": file.id, "tools": [{"type": "code_interpreter"}]}
128 ],
129 }
130 ]
131)
132```
133
134```bash134```bash
135curl https://api.openai.com/v1/threads \135curl https://api.openai.com/v1/threads \
136 -H "Authorization: Bearer $OPENAI_API_KEY" \136 -H "Authorization: Bearer $OPENAI_API_KEY" \
161 161
162Tools cannot access image content unless specified. To pass image files to Code Interpreter, add the file ID in the message `attachments` list to allow the tool to read and analyze the input. Image URLs cannot be downloaded in Code Interpreter today.162Tools cannot access image content unless specified. To pass image files to Code Interpreter, add the file ID in the message `attachments` list to allow the tool to read and analyze the input. Image URLs cannot be downloaded in Code Interpreter today.
163 163
164```python
165file = client.files.create(file=open("myimage.png", "rb"), purpose="vision")
166thread = client.beta.threads.create(
167 messages=[
168 {
169 "role": "user",
170 "content": [
171 {
172 "type": "text",
173 "text": "What is the difference between these images?",
174 },
175 {
176 "type": "image_url",
177 "image_url": {
178 "url": "https://openai-documentation.vercel.app/images/cat_and_otter.png"
179 },
180 },
181 {"type": "image_file", "image_file": {"file_id": file.id}},
182 ],
183 }
184 ]
185)
186```
187
188```javascript164```javascript
189import fs from "fs";165import fs from "fs";
190 166
217});193});
218```194```
219 195
196```python
197file = client.files.create(file=open("myimage.png", "rb"), purpose="vision")
198thread = client.beta.threads.create(
199 messages=[
200 {
201 "role": "user",
202 "content": [
203 {
204 "type": "text",
205 "text": "What is the difference between these images?",
206 },
207 {
208 "type": "image_url",
209 "image_url": {
210 "url": "https://openai-documentation.vercel.app/images/cat_and_otter.png"
211 },
212 },
213 {"type": "image_file", "image_file": {"file_id": file.id}},
214 ],
215 }
216 ]
217)
218```
219
220```bash220```bash
221# Upload a file with an "vision" purpose221# Upload a file with an "vision" purpose
222curl https://api.openai.com/v1/files \222curl https://api.openai.com/v1/files \
261- `low` will enable the "low res" mode. The model will receive a low-res 512px x 512px version of the image, and represent the image with a budget of 85 tokens. This allows the API to return faster responses and consume fewer input tokens for use cases that do not require high detail.261- `low` will enable the "low res" mode. The model will receive a low-res 512px x 512px version of the image, and represent the image with a budget of 85 tokens. This allows the API to return faster responses and consume fewer input tokens for use cases that do not require high detail.
262- `high` will enable "high res" mode, which first allows the model to see the low res image and then creates detailed crops of input images based on the input image size. Use the [pricing calculator](https://openai.com/api/pricing/) to see token counts for various image sizes.262- `high` will enable "high res" mode, which first allows the model to see the low res image and then creates detailed crops of input images based on the input image size. Use the [pricing calculator](https://openai.com/api/pricing/) to see token counts for various image sizes.
263 263
264```python
265thread = client.beta.threads.create(
266 messages=[
267 {
268 "role": "user",
269 "content": [
270 {"type": "text", "text": "What is this an image of?"},
271 {
272 "type": "image_url",
273 "image_url": {
274 "url": "https://openai-documentation.vercel.app/images/cat_and_otter.png",
275 "detail": "high",
276 },
277 },
278 ],
279 }
280 ]
281)
282```
283
284```javascript264```javascript
285const thread = await openai.beta.threads.create({265const thread = await openai.beta.threads.create({
286 messages: [266 messages: [
304});284});
305```285```
306 286
287```python
288thread = client.beta.threads.create(
289 messages=[
290 {
291 "role": "user",
292 "content": [
293 {"type": "text", "text": "What is this an image of?"},
294 {
295 "type": "image_url",
296 "image_url": {
297 "url": "https://openai-documentation.vercel.app/images/cat_and_otter.png",
298 "detail": "high",
299 },
300 },
301 ],
302 }
303 ]
304)
305```
306
307```bash307```bash
308curl https://api.openai.com/v1/threads \308curl https://api.openai.com/v1/threads \
309 -H "Authorization: Bearer $OPENAI_API_KEY" \309 -H "Authorization: Bearer $OPENAI_API_KEY" \
415 415
416When you have all the context you need from your user in the Thread, you can run the Thread with an Assistant of your choice.416When you have all the context you need from your user in the Thread, you can run the Thread with an Assistant of your choice.
417 417
418```javascript
419const run = await openai.beta.threads.runs.create(thread.id, {
420 assistant_id: assistant.id,
421});
422```
423
418```python424```python
419run = client.beta.threads.runs.create(425run = client.beta.threads.runs.create(
420 thread_id=thread.id,426 thread_id=thread.id,
422)428)
423```429```
424 430
425```javascript
426const run = await openai.beta.threads.runs.create(thread.id, {
427 assistant_id: assistant.id,
428});
429```
430
431```bash431```bash
432curl https://api.openai.com/v1/threads/THREAD_ID/runs \432curl https://api.openai.com/v1/threads/THREAD_ID/runs \
433 -H "Authorization: Bearer $OPENAI_API_KEY" \433 -H "Authorization: Bearer $OPENAI_API_KEY" \
441 441
442By default, a Run will use the `model` and `tools` configuration specified in Assistant object, but you can override most of these when creating the Run for added flexibility:442By default, a Run will use the `model` and `tools` configuration specified in Assistant object, but you can override most of these when creating the Run for added flexibility:
443 443
444```javascript
445const run = await openai.beta.threads.runs.create(thread.id, {
446 assistant_id: assistant.id,
447 model: "gpt-4o",
448 instructions: "New instructions that override the Assistant instructions",
449 tools: [{ type: "code_interpreter" }, { type: "file_search" }],
450});
451```
452
444```python453```python
445run = client.beta.threads.runs.create(454run = client.beta.threads.runs.create(
446 thread_id=thread.id,455 thread_id=thread.id,
451)460)
452```461```
453 462
454```javascript
455const run = await openai.beta.threads.runs.create(thread.id, {
456 assistant_id: assistant.id,
457 model: "gpt-4o",
458 instructions: "New instructions that override the Assistant instructions",
459 tools: [{ type: "code_interpreter" }, { type: "file_search" }],
460});
461```
462
463```bash463```bash
464curl https://api.openai.com/v1/threads/THREAD_ID/runs \464curl https://api.openai.com/v1/threads/THREAD_ID/runs \
465 -H "Authorization: Bearer $OPENAI_API_KEY" \465 -H "Authorization: Bearer $OPENAI_API_KEY" \