RAG Multimodal Complexe Financial Reports

RAG_multimodal_complexe_financial_reports
March 17, 2024
[ ]: !pip install llama-index

!pip install llama-index-core
!pip install llama-index-llms-openai
!pip install llama-index-multi-modal-llms-openai
!pip install llama-index-llms-gemini
!pip install llama-index-multi-modal-llms-gemini
!pip install llama-index-embeddings-gemini
!pip install llama-parse
[ ]: from google.colab import userdata

LLAMA_CLOUD_API_KEY = userdata.get('LLAMAPARSE_API_KEY')
GEMINI_API_KEY = userdata.get('GEMINI_API_KEY')
OPENAI_API_KEY = userdata.get('OPENAI_API_KEY')
import openai
openai.api_key = OPENAI_API_KEY
from llama_index.multi_modal_llms.gemini import GeminiMultiModal

gemini_pro = GeminiMultiModal(model_name="models/
↪gemini-pro-vision",api_key=GEMINI_API_KEY)
[ ]: from llama_index.llms.gemini import Gemini

from llama_index.embeddings.gemini import GeminiEmbedding
from llama_index.core import Settings
Settings.llm = Gemini(api_key=GEMINI_API_KEY)
Settings.embed_model = GeminiEmbedding(
model_name="models/embedding-001", api_key=GEMINI_API_KEY
)
from llama_index.multi_modal_llms.openai import OpenAIMultiModal

openai_mm_llm = OpenAIMultiModal(
model="gpt-4-vision-preview", api_key=OPENAI_API_KEY, max_new_tokens=1500
)
1
1 PDF with charts and tables as a text: JPMorgan Annual Report
[ ]: local_path='./'
file_path = local_path + 'jpmorgan_annualreport-2022-splitted.pdf'
1.1 LlamaParse with JSON mode:

I used the JSON mode to see exactly how the data in the charts are extracted. As you can see,
they are empty. Additionally, I wanted to confirm that they are not considered as images, which is
why the image object is empty.
[ ]: from llama_parse import LlamaParse
parser = LlamaParse(api_key=LLAMA_CLOUD_API_KEY, verbose=True)

json_objs = parser.get_json_result(file_path)
json_list = json_objs[0]["pages"]
json_list
[ ]: from llama_index.core.schema import TextNode

from typing import List
def get_text_nodes(json_list: List[dict]):

text_nodes = []
for idx, page in enumerate(json_list):
text_node = TextNode(
text=page["text"],
metadata={
"page": page["page"]
}
)
text_nodes.append(text_node)
return text_nodes
[ ]: text_nodes = get_text_nodes(json_list)
[ ]: text_nodes
1.2 LlamaParse: Extraction as a Markdown format

1.2.1 Text as Markdown
[ ]: from llama_parse import LlamaParse
parser = LlamaParse(api_key=LLAMA_CLOUD_API_KEY, verbose=True,␣

↪result_type="markdown")
# use SimpleDirectoryReader to parse our file
2
file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader(input_files=[file_path],␣
↪file_extractor=file_extractor).load_data()
print(documents[0].text)
Started parsing the file under job_id 402894ac-0560-4240-a915-0c12599a273a

## Earnings, Diluted Earnings per Share and Return on Tangible Common Equity
| |Reported|Excluding reserve release/build|

|---|---|---|
| |2020|2021|2022|2020|2021|2022|
|Net income ($B)|$29.1|$48.3|$37.7|$38.4|$39.1|$40.4|
|Diluted EPS ($)|$8.88|$15.36|$12.09|$11.87|$12.35|$12.99|
|ROTCE|14.4%|23.0%|17.7%|19.3%|18.5%|19.1%|
| |2004|2005|2006|2007|2008|2009|2010|2011|2012|2013|2014|2015|2016|2017|2018|20
19|2020|2021|2022|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
|
| | | | | | | | | | | | | | | |Net income| | | | | | | | | | | | | | | | | | | |
|
| | | |Diluted earnings per share (EPS)| | | | | | | | | | | | | | | | | | | | |
| | | | | | | |Return on tangible common equity (ROTCE)| | | | | | | | | | | | |
| | | | | | | |
|Adjusted ROTCE| | | | | | | | | | | | | | | | | | | | |
1 Firmwide results excluding reserve release/build are non-GAAP financial

measures.
2 Adjusted net income excludes $2.4 billion from net income in 2017 as a result
of the enactment of the Tax Cuts and Jobs Act.
GAAP = Generally accepted accounting principles
ROTCE = Return on tangible common equity
An important note to describe why we are showing the table above: The loan loss
reserve accounting rules — which are life-of-loan estimated losses based upon
probability-based economic scenarios — generate huge swings in earnings that can
be unrelated to actual credit performance. This was particularly true for the
COVID-19 years when, during the first six months of the pandemic, we built
approximately $16 billion in reserves. Then in the next six quarters, we
released essentially the equivalent number. We did so only because the scenarios
used to estimate future credit losses changed dramatically. The table above
shows reported net income, with and without loan loss reserve changes.
Throughout this period, the credit portfolio was healthy, and charge-offs
remained below pre-pandemic levels. Either way, the company had strong absolute
3
and relative performance.
---
|Tangible Book Value|Average Stock Price per Share|
|---|---|
|$155.61|2004–2022|
|$110.72|$113.80|
|$128.13| |
|$106.52| |
|High: $169.81|Low: $101.28|
|$92.01| |
|$58.17|$63.83|
|$65.62| |
|$66.11|$71.53|
|$73.12| |
|$38.70|$43.93|
|$47.75| |
|$39.83|$40.36|
|$39.36|$39.22|
|$51.88|$51.44|
|$53.56|$56.33|
|$60.98| |
|$36.07| |
|$35.49| |
|$38.68|$40.72|
|$44.60|$48.13|
|$33.62| |
|$27.09|$30.12|
|$15.35|$16.45|
|$18.88|$21.96|
|$22.52| |
Tangible book value Average stock price
1 9% compound annual growth rate since 2004.
Stock total return analysis
| |Bank One|S&P 500 Index|S&P Financials Index|

|---|---|---|---|
|Performance since becoming CEO of Bank One|Compounded annual
gain|11.3%|6.1%|4.6%|
|Overall gain|1,047.8%|287.7%|176.1%|
| |JPMorgan Chase & Co.|S&P 500 Index|S&P Financials Index|
|Performance since the Bank One and JPMorgan Chase & Co. merger|Compounded
annual gain|9.9%|8.9%|4.4%|
|Overall gain|471.6%|386.8%|120.0%|
|Performance for the period ended December 31, 2022|Compounded annual
gain/(loss)|One year|(12.6)%|(18.1)%|(10.5)%|
4
| |Five years|7.7%|9.4%|6.4%|
| |Ten years|14.9%|12.6%|12.1%|
This chart shows actual returns of the stock, with dividends reinvested, for
heritage shareholders of Bank One and JPMorgan Chase & Co. vs. the Standard &
Poor’s 500 Index (S&P 500 Index) and the Standard & Poor’s Financials Index (S&P
Financials Index).
1 On March 27, 2000, Jamie Dimon was hired as CEO of Bank One.
1.2.2 Querying
[ ]: index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
# query the engine
query = "What is the diluted EPS value in 2021 when Reported"
response = query_engine.query(query)
print(response)
$15.36
[ ]: query = "What is the diluted EPS value in 2021 when Excluding reserve release/
↪build1"
print(response)
$12.35
[ ]: query = "Compounded annual gain"

print(response)
9.9%
[ ]: query = "What is the Compounded annual gain or loss of the SP500 Index since␣
↪JPMorgan and Bank One have merged?"
print(response)
8.9%
[ ]: query = "What is the ten years Compounded annual gain or loss of the SP500␣
↪Financials Index?"
print(response)
12.1%
5
2 PDF as images
2.1 Save each page as an Image
[ ]: local_path='./'
file_path_image = local_path + 'jpmorgan_annualreport-2022-splitted.pdf'
[ ]: import fitz
from PIL import Image
import matplotlib.pyplot as plt
import os
[ ]: # Open the PDF file

pdf_document = fitz.open(file_path_image)
output_directory_path = local_path+'images2'
# Iterate through each page and convert to an image
for page_number in range(pdf_document.page_count):
page = pdf_document[page_number]
pix = page.get_pixmap()
# Create a Pillow Image object from the pixmap
image = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
# Save the image
image.save(f"{output_directory_path}/page_{page_number + 1}.png")
# Close the PDF file

pdf_document.close()
[ ]: from PIL import Image

import os
image_paths = []
for img_path in os.listdir(f"{output_directory_path}"):
image_paths.append(str(os.path.join(f"{output_directory_path}", img_path)))
def plot_images(image_paths):
images_shown = 0
plt.figure(figsize=(16, 9))
for img_path in image_paths:
if os.path.isfile(img_path):
image = Image.open(img_path)
plt.subplot(3, 3, images_shown + 1)
plt.imshow(image)
plt.xticks([])
6
plt.yticks([])
images_shown += 1
if images_shown >= 9:
break
plot_images(image_paths[:])
[ ]: from llama_index.core import SimpleDirectoryReader
output_directory_path = local_path+'images'
# Load images as ImageDocument
documents_images_v2 = SimpleDirectoryReader(f"{output_directory_path}/").
↪load_data()
2.2 Calling Gemini-pro-vision

[ ]: # from llama_index.multi_modal_llms.gemini import GeminiMultiModal
# gemini_pro = GeminiMultiModal(model_name="models/
Trying to get a json format from the model

[ ]: image_prompt = """
Please load the tables and charts data and output in the json format from␣
↪the image.
Please try your best to extract the table data from the image.
If you can't extract the table data, please summarize image and return the␣
↪summary.
"""
7
response = gemini_pro.complete(
prompt="Describe the images as an alternative text",
image_documents=documents_images_v2,
)
text=str(response)
)
[ ]: text_node
[ ]: TextNode(id_='0b3194b4-1f14-4538-a6f0-b652bf7e08e5', embedding=None,
metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[],
relationships={}, text=' The first image is a table that shows the earnings,
diluted earnings per share, and return on tangible common equity of Bank One
from 2004 to 2022. The second image is a graph that shows the tangible book
value and average stock price of Bank One from 2004 to 2022. The third image is
a table that shows the stock total return analysis of Bank One from 2004 to
2022.', start_char_idx=None, end_char_idx=None,
text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}',
metadata_seperator='\n')
Trying to get a markdown format from the model

Please load the charts data from the images and output it in markdown␣
↪format .
Please try your best to extract the charts data from the images.
If you can't extract the chart data, please summarize image and return the␣
↪summary.
"""
response2 = gemini_pro.complete(
prompt=image_prompt,
)
text=str(response2)
)
[ ]: text_node
[ ]: TextNode(id_='3ab2bfcd-05dc-4bd1-a4fe-24f930e51c64', embedding=None,
metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[],
relationships={}, text=' ```json\n{\n "tables": [\n {\n "title":
"Earnings, Diluted Earnings per Share and Return on Tangible Common Equity",\n
8
"headers": [\n "Reported",\n "Excluding reserve release/hold"\n
],\n "rows": [\n {\n "Net income": [\n 24.83,\n
35.93,\n 39.18,\n 53.84,\n 120.29\n
],\n "Diluted EPS": [\n 1.52,\n 2.07,\n
2.26,\n 3.02,\n 6.26\n ],\n "Return on
tangible common equity": [\n 10.2%,\n 13.5%,\n
15.5%,\n 18.9%,\n 23.2%\n ]\n }\n ]\n
},\n {\n "title": "Tangible Book Value and Average Stock Price per
Share",\n "headers": [\n "Tangible book value",\n "Average
stock price"\n ],\n "rows": [\n {\n "2004": [\n
36.07,\n 35.49\n ],\n "2005": [\n
38.70,\n 42.93\n ],\n "2006": [\n
43.75,\n 43.39\n ],\n "2007": [\n
49.36,\n 39.32\n ],\n "2008": [\n
55.83,\n 44.60\n ],\n "2009": [\n
63.83,\n 58.17\n ],\n "2010": [\n
66.62,\n 65.64\n ],\n "2011": [\n
58.18,\n 55.12\n ],\n "2012": [\n
51.88,\n 35.49\n ],\n "2013": [\n
54.06,\n 44.13\n ],\n "2014": [\n
60.98,\n 66.11\n ],\n "2015": [\n
71.53,\n 73.12\n ],\n "2016": [\n
84.13,\n 92.01\n ],\n "2017": [\n
104.52,\n 113.80\n ],\n "2018": [\n
118.34,\n 128.13\n ],\n "2019": [\n
135.35,\n 155.61\n ],\n "2020": [\n
128.13,\n 109.22\n ],\n "2021": [\n
138.62,\n 120.09\n ],\n "2022": [\n
155.61,\n 132.83\n ]\n }\n ]\n }\n ],\n
"charts": [\n {\n "title": "Earnings, Diluted Earnings per Share and
Return on Tangible Common Equity",\n "data": [\n {\n
"name": "Net income",\n "values": [\n 24.83,\n
35.93,\n 39.18,\n 53.84,\n 120.29\n
]\n },\n {\n "name": "Diluted EPS",\n "values":
[\n 1.52,\n 2.07,\n 2.26,\n 3.02,\n
6.26\n ]\n },\n {\n "name": "Return on tangible
common equity",\n "values": [\n 10.2,\n 13.5,\n
15.5,\n 18.9,\n 23.2', start_char_idx=None,
end_char_idx=None, text_template='{metadata_str}\n\n{content}',
metadata_template='{key}: {value}', metadata_seperator='\n')
[ ]: response2.text
[ ]: ' ```json\n{\n "tables": [\n {\n "title": "Earnings, Diluted Earnings

per Share and Return on Tangible Common Equity",\n "headers": [\n
"Reported",\n "Excluding reserve release/hold"\n ],\n "rows":
[\n {\n "Net income": [\n 24.83,\n
9
35.93,\n 39.18,\n 53.84,\n 120.29\n
],\n "Diluted EPS": [\n 1.52,\n 2.07,\n
2.26,\n 3.02,\n 6.26\n ],\n "Return on
tangible common equity": [\n 10.2%,\n 13.5%,\n
15.5%,\n 18.9%,\n 23.2%\n ]\n }\n ]\n
},\n {\n "title": "Tangible Book Value and Average Stock Price per
Share",\n "headers": [\n "Tangible book value",\n "Average
stock price"\n ],\n "rows": [\n {\n "2004": [\n
36.07,\n 35.49\n ],\n "2005": [\n
38.70,\n 42.93\n ],\n "2006": [\n
43.75,\n 43.39\n ],\n "2007": [\n
49.36,\n 39.32\n ],\n "2008": [\n
55.83,\n 44.60\n ],\n "2009": [\n
63.83,\n 58.17\n ],\n "2010": [\n
66.62,\n 65.64\n ],\n "2011": [\n
58.18,\n 55.12\n ],\n "2012": [\n
51.88,\n 35.49\n ],\n "2013": [\n
54.06,\n 44.13\n ],\n "2014": [\n
60.98,\n 66.11\n ],\n "2015": [\n
71.53,\n 73.12\n ],\n "2016": [\n
84.13,\n 92.01\n ],\n "2017": [\n
104.52,\n 113.80\n ],\n "2018": [\n
118.34,\n 128.13\n ],\n "2019": [\n
135.35,\n 155.61\n ],\n "2020": [\n
128.13,\n 109.22\n ],\n "2021": [\n
138.62,\n 120.09\n ],\n "2022": [\n
155.61,\n 132.83\n ]\n }\n ]\n }\n ],\n
"charts": [\n {\n "title": "Earnings, Diluted Earnings per Share and
Return on Tangible Common Equity",\n "data": [\n {\n
"name": "Net income",\n "values": [\n 24.83,\n
35.93,\n 39.18,\n 53.84,\n 120.29\n
]\n },\n {\n "name": "Diluted EPS",\n "values":
[\n 1.52,\n 2.07,\n 2.26,\n 3.02,\n
6.26\n ]\n },\n {\n "name": "Return on tangible
common equity",\n "values": [\n 10.2,\n 13.5,\n
15.5,\n 18.9,\n 23.2'
Please load the charts and table data from the images and output it in a␣
↪json format .
↪summary.
"""
response_gemini = gemini_pro.complete(
10
)
# text_node_gmeini = TextNode(
# text=str(response_gemini)
# )
[ ]: response_gemini.text
[ ]: ' ```json\n{\n "Earnings per share": [\n {\n "year": 2004,\n

"reported": 4.52,\n "excluding reserve release": 4.52\n },\n {\n
"year": 2005,\n "reported": 5.06,\n "excluding reserve release":
5.06\n },\n {\n "year": 2006,\n "reported": 6.14,\n
"excluding reserve release": 6.14\n },\n {\n "year": 2007,\n
1.44\n },\n {\n "year": 2009,\n "reported": -1.32,\n
"excluding reserve release": -1.32\n },\n {\n "year": 2010,\n
"reported": 29.22,\n "excluding reserve release": 29.22\n }\n ],\n
"Return on tangible common equity": [\n {\n "year": 2004,\n
"reported": 19%,\n "excluding reserve release": 19%\n },\n {\n
"year": 2005,\n "reported": 22%,\n "excluding reserve release": 22%\n
},\n {\n "year": 2006,\n "reported": 26%,\n "excluding reserve
release": 26%\n },\n {\n "year": 2007,\n "reported": 27%,\n
"excluding reserve release": 27%\n },\n {\n "year": 2008,\n
"reported": -10%,\n "excluding reserve release": 6%\n },\n {\n
"year": 2009,\n "reported": -11%,\n "excluding reserve release": -'
[ ]: response_gemini
11
[ ]: CompletionResponse(text=' ```json\n{\n "Earnings per share": [\n {\n
"reported": -1.32,\n "excluding reserve release": -1.32\n },\n {\n
29.22\n }\n ],\n "Return on tangible common equity": [\n {\n
"year": 2008,\n "reported": -10%,\n "excluding reserve release": 6%\n
},\n {\n "year": 2009,\n "reported": -11%,\n "excluding
reserve release": -', additional_kwargs={}, raw={'content': {'parts': [{'text':
' ```json\n{\n "Earnings per share": [\n {\n "year": 2004,\n
1.44\n },\n {\n "year": 2009,\n "reported": -1.32,\n
"excluding reserve release": -1.32\n },\n {\n "year": 2010,\n
12
"reported": 29.22,\n "excluding reserve release": 29.22\n }\n ],\n
"Return on tangible common equity": [\n {\n "year": 2004,\n
"reported": -10%,\n "excluding reserve release": 6%\n },\n {\n
"year": 2009,\n "reported": -11%,\n "excluding reserve release": -'}],
'role': 'model'}, 'finish_reason': 1, 'index': 0, 'safety_ratings':
[{'category': 9, 'probability': 1, 'blocked': False}, {'category': 8,
'probability': 1, 'blocked': False}, {'category': 7, 'probability': 1,
'blocked': False}, {'category': 10, 'probability': 1, 'blocked': False}],
'token_count': 0, 'grounding_attributions': [], 'block_reason': 0},
logprobs=None, delta=None)
[ ]:
2.3 Images to Multimodal

output_directory_path = local_path+'images'
↪load_data()
2.4 Gemini pro

↪format .
13
↪summary.
"""
response2 = gemini_pro.complete(
)
text=str(response2)
)
[ ]: from llama_index.multi_modal_llms.gemini import GeminiMultiModal

gemini_pro = GeminiMultiModal(model_name="models/
[ ]: response2
[ ]: CompletionResponse(text=' | Year | Net income reported | Excluding reserve

release/ (loss) | Diluted EPS | Excluding reserve release/(loss)
|\n|---|---|---|---|---|\n| 2022 | 24.81 B | 38.9 B | 5.98 | 9.44 |\n| 2021 |
23.54 B | 27.0 B | 5.67 | 6.56 |\n| 2020 | (12.60) B | 1.0 B | (3.02) | 0.24
|\n| 2019 | 10.26 B | 12.9 B | 2.47 | 3.12 |\n| 2018 | 26.45 B | 29.2 B | 6.36 |
7.06 |\n| 2017 | 22.26 B | 23.6 B | 5.33 | 5.68 |\n| 2016 | 17.86 B | 21.2 B |
4.29 | 5.11 |\n| 2015 | 16.58 B | 18.8 B | 4.01 | 4.57 |\n| 2014 | 15.55 B |
18.3 B | 3.77 | 4.44 |\n| 2013 | 16.45 B | 18.2 B | 4.01 | 4.42 |\n| 2012 |
17.44 B | 19.2 B | 4.24 | 4.69 |\n| 2011 | 15.35 B | 16.9 B | 3.72 | 4.11 |\n|
2010 | 14.99 B | 16.8 B | 3.63 | 4.08 |\n| 2009 | 11.33 B | 13.5 B | 2.76 | 3.29
|\n| 2008 | 10.67 B | 12.7 B | 2.62 | 3.12 |\n| 2007 | 16.45 B | 18.8 B | 4.03 |
4.62 |\n| 2006 | 13.57 B | 15.2 B | 3.31 | 3.73 |\n| 2005 | 11.34 B | 13.2 B |
2.79 | 3.26 |\n| 2004 | 8.53 B | 10.2 B | 2.13 | 2.56 |\n\n\n| Year | Tangible
book value | Average stock price |\n|---|---|---|\n| 2022 | 73.12 B | 67.92 |\n|
2021 | 65.62 B | 55.18 |\n| 2020 | 58.13 B | 44.10 |\n| 2019 | 51.88 B | 66.41
|\n| 2018 | 43.83 B | 71.53 |\n| 2017 | 35.98 B | 63.64 |\n| 2016 | 33.83 B |
58.23 |\n| 2015 | 32.09 B | 48.13 |\n| 2014 | 29.36 B | 40.36 |\n| 2013 | 27.09
B | 39.32 |\n| 2012 | 25.18 B | 32.12 |\n| 2011 | 22.96 B | 33.49 |\n| 2010 |
21.09 B | 27.09 |\n| 2009 | ', additional_kwargs={}, raw={'content': {'parts':
[{'text': ' | Year | Net income reported | Excluding reserve release/ (loss) |
Diluted EPS | Excluding reserve release/(loss) |\n|---|---|---|---|---|\n| 2022
| 24.81 B | 38.9 B | 5.98 | 9.44 |\n| 2021 | 23.54 B | 27.0 B | 5.67 | 6.56 |\n|
2020 | (12.60) B | 1.0 B | (3.02) | 0.24 |\n| 2019 | 10.26 B | 12.9 B | 2.47 |
3.12 |\n| 2018 | 26.45 B | 29.2 B | 6.36 | 7.06 |\n| 2017 | 22.26 B | 23.6 B |
5.33 | 5.68 |\n| 2016 | 17.86 B | 21.2 B | 4.29 | 5.11 |\n| 2015 | 16.58 B |
18.8 B | 4.01 | 4.57 |\n| 2014 | 15.55 B | 18.3 B | 3.77 | 4.44 |\n| 2013 |
16.45 B | 18.2 B | 4.01 | 4.42 |\n| 2012 | 17.44 B | 19.2 B | 4.24 | 4.69 |\n|
2011 | 15.35 B | 16.9 B | 3.72 | 4.11 |\n| 2010 | 14.99 B | 16.8 B | 3.63 | 4.08
14
|\n| 2009 | 11.33 B | 13.5 B | 2.76 | 3.29 |\n| 2008 | 10.67 B | 12.7 B | 2.62 |
3.12 |\n| 2007 | 16.45 B | 18.8 B | 4.03 | 4.62 |\n| 2006 | 13.57 B | 15.2 B |
3.31 | 3.73 |\n| 2005 | 11.34 B | 13.2 B | 2.79 | 3.26 |\n| 2004 | 8.53 B | 10.2
B | 2.13 | 2.56 |\n\n\n| Year | Tangible book value | Average stock price
|\n|---|---|---|\n| 2022 | 73.12 B | 67.92 |\n| 2021 | 65.62 B | 55.18 |\n| 2020
| 58.13 B | 44.10 |\n| 2019 | 51.88 B | 66.41 |\n| 2018 | 43.83 B | 71.53 |\n|
2017 | 35.98 B | 63.64 |\n| 2016 | 33.83 B | 58.23 |\n| 2015 | 32.09 B | 48.13
|\n| 2014 | 29.36 B | 40.36 |\n| 2013 | 27.09 B | 39.32 |\n| 2012 | 25.18 B |
32.12 |\n| 2011 | 22.96 B | 33.49 |\n| 2010 | 21.09 B | 27.09 |\n| 2009 | '}],
'role': 'model'}, 'finish_reason': 1, 'index': 0, 'safety_ratings':
[{'category': 9, 'probability': 1, 'blocked': False}, {'category': 8,
'probability': 1, 'blocked': False}, {'category': 7, 'probability': 1,
'blocked': False}, {'category': 10, 'probability': 1, 'blocked': False}],
'token_count': 0, 'grounding_attributions': [], 'block_reason': 0},
2.5 GPT-4-Vision
2.5.1 Prompt1: format markdown
↪format .
↪summary.
"""
response_gpt= openai_mm_llm.complete(
)
text_node_gpt = TextNode(
text=str(response_gpt)
)
[ ]: response_gpt
[ ]: CompletionResponse(text='I\'m unable to extract the exact numerical data from

the images to output in markdown format. However, I can provide a summary of the
content depicted in the images.\n\n### Image 1 Summary:\nThe first image appears
to be a financial chart from a corporate report, showing "Earnings, Diluted
Earnings per Share and Return on Tangible Common Equity" from 2012 to 2022. The
chart is a combination of bar and line graphs. The bar graph shows earnings in
billions of dollars, while the line graph indicates the return on tangible
common equity (ROTCE) as a percentage. The earnings are shown to be increasing
15
over the years, with a dip around 2020, which could be attributed to the
COVID-19 impact. The ROTCE also shows an upward trend with some fluctuations,
peaking in the most recent year displayed.\n\n### Image 2 Summary:\nThe second
image is another financial chart titled "Tangible Book Value and Average Stock
Price per Share" from 2004 to 2022. This chart includes two line graphs, one
representing the tangible book value per share and the other showing the average
stock price per share. Both metrics show a general upward trend over the years.
The tangible book value per share starts at around $35 in 2004 and increases to
over $80 by 2022. The average stock price per share begins at approximately $50
in 2004 and grows to over $150 by 2022, with some volatility in between.\n\nBoth
images are part of a financial analysis and are likely intended to communicate
the company\'s financial performance and growth to stakeholders.',
additional_kwargs={}, raw={'id': 'chatcmpl-93iQszYjzt8LiZeslSMlWLOADY7IN',
'choices': [Choice(finish_reason='stop', index=0, logprobs=None,
message=ChatCompletionMessage(content='I\'m unable to extract the exact
numerical data from the images to output in markdown format. However, I can
provide a summary of the content depicted in the images.\n\n### Image 1
Summary:\nThe first image appears to be a financial chart from a corporate
report, showing "Earnings, Diluted Earnings per Share and Return on Tangible
Common Equity" from 2012 to 2022. The chart is a combination of bar and line
graphs. The bar graph shows earnings in billions of dollars, while the line
graph indicates the return on tangible common equity (ROTCE) as a percentage.
The earnings are shown to be increasing over the years, with a dip around 2020,
which could be attributed to the COVID-19 impact. The ROTCE also shows an upward
trend with some fluctuations, peaking in the most recent year displayed.\n\n###
Image 2 Summary:\nThe second image is another financial chart titled "Tangible
Book Value and Average Stock Price per Share" from 2004 to 2022. This chart
includes two line graphs, one representing the tangible book value per share and
the other showing the average stock price per share. Both metrics show a general
upward trend over the years. The tangible book value per share starts at around
$35 in 2004 and increases to over $80 by 2022. The average stock price per share
begins at approximately $50 in 2004 and grows to over $150 by 2022, with some
volatility in between.\n\nBoth images are part of a financial analysis and are
likely intended to communicate the company\'s financial performance and growth
to stakeholders.', role='assistant', function_call=None, tool_calls=None))],
'created': 1710672810, 'model': 'gpt-4-1106-vision-preview', 'object':
'chat.completion', 'system_fingerprint': None, 'usage':
CompletionUsage(completion_tokens=326, prompt_tokens=226, total_tokens=552)},
[ ]: print(response_gpt.text)
I'm unable to extract the exact numerical data from the images to output in
markdown format. However, I can provide a summary of the content depicted in the
images.
### Image 1 Summary:
16
The first image appears to be a financial chart from a corporate report, showing
"Earnings, Diluted Earnings per Share and Return on Tangible Common Equity" from
2012 to 2022. The chart is a combination of bar and line graphs. The bar graph
shows earnings in billions of dollars, while the line graph indicates the return
on tangible common equity (ROTCE) as a percentage. The earnings are shown to be
increasing over the years, with a dip around 2020, which could be attributed to
the COVID-19 impact. The ROTCE also shows an upward trend with some
fluctuations, peaking in the most recent year displayed.
### Image 2 Summary:

The second image is another financial chart titled "Tangible Book Value and
Average Stock Price per Share" from 2004 to 2022. This chart includes two line
graphs, one representing the tangible book value per share and the other showing
the average stock price per share. Both metrics show a general upward trend over
the years. The tangible book value per share starts at around $35 in 2004 and
increases to over $80 by 2022. The average stock price per share begins at
approximately $50 in 2004 and grows to over $150 by 2022, with some volatility
in between.
Both images are part of a financial analysis and are likely intended to
communicate the company's financial performance and growth to stakeholders.
[ ]: # I'm unable to extract the exact numerical data from the images to output in␣
↪markdown format. However, I can provide a summary of the content depicted in␣
↪the images.
# ### Image 1 Summary:

# The first image appears to be a financial chart from a corporate report,␣
↪showing "Earnings, Diluted Earnings per Share and Return on Tangible Common␣
↪Equity" from 2012 to 2022.
# The chart is a combination of bar and line graphs.

# The bar graph shows earnings in billions of dollars, while the line graph␣
↪indicates the return on tangible common equity (ROTCE) as a percentage.
# The earnings are shown to be increasing over the years, with a dip around␣
↪2020, which could be attributed to the COVID-19 impact.
# The ROTCE also shows an upward trend with some fluctuations, peaking in the␣
↪most recent year displayed.
# ### Image 2 Summary:

# The second image is another financial chart titled "Tangible Book Value and␣
↪Average Stock Price per Share" from 2004 to 2022.
# This chart includes two line graphs, one representing the tangible book value␣
↪per share and the other showing the average stock price per share.
# Both metrics show a general upward trend over the years. The tangible book␣
↪value per share starts at around $35 in 2004 and increases to over $80 by␣
↪2022.
17
# The average stock price per share begins at approximately $50 in 2004 and␣
↪grows to over $150 by 2022, with some volatility in between.
# Both images are part of a financial analysis and are likely intended to␣
↪communicate the company's financial performance and growth to stakeholders.
2.5.2 Prompt2: specifc question
↪format .
↪summary.
Additionnaly, answer to this question: What was the percentage of the␣

↪return on tangible common equity (ROTCE) in in 2012"
"""
response_gpt2= openai_mm_llm.complete(
)
[ ]: response_gpt2
[ ]: CompletionResponse(text="I'm sorry, but I'm unable to extract specific chart

data from images. However, I can provide a summary of the content depicted in
the images.\n\nThe first image appears to be a financial chart showing Earnings,
Diluted Earnings per Share, and Return on Tangible Common Equity (ROTCE) from
2012 to 2022. The chart is divided into three sections, each representing a
different metric. The top section shows net income in billions, the middle
section shows diluted earnings per share, and the bottom section shows ROTCE
percentages. The chart includes blue and white bars, with the blue bars
representing the actual figures and the white bars possibly representing
adjusted figures or projections. There are also annotations on the chart that
provide additional context or explanations for certain years' figures.\n\nThe
second image is another financial chart that presents the Tangible Book Value
and Average Stock Price per Share from 2004 to 2022. The chart shows two lines:
one for the tangible book value per share and another for the average stock
price. The chart indicates a general upward trend in both metrics over the
years.\n\nRegarding the specific question about the percentage of the Return on
Tangible Common Equity (ROTCE) in 2012, the chart in the first image should
contain this information. However, due to the limitations of my current
capabilities, I cannot extract the exact percentage from the image. To obtain
this information, you would need to refer to the source of the financial data or
the original document from which the image was taken.", additional_kwargs={},
18
raw={'id': 'chatcmpl-93iYC1g3XesZcyYMbiqgMIAufPWmI', 'choices':
[Choice(finish_reason='stop', index=0, logprobs=None,
message=ChatCompletionMessage(content="I'm sorry, but I'm unable to extract
specific chart data from images. However, I can provide a summary of the content
depicted in the images.\n\nThe first image appears to be a financial chart
showing Earnings, Diluted Earnings per Share, and Return on Tangible Common
Equity (ROTCE) from 2012 to 2022. The chart is divided into three sections, each
representing a different metric. The top section shows net income in billions,
the middle section shows diluted earnings per share, and the bottom section
shows ROTCE percentages. The chart includes blue and white bars, with the blue
bars representing the actual figures and the white bars possibly representing
adjusted figures or projections. There are also annotations on the chart that
provide additional context or explanations for certain years' figures.\n\nThe
second image is another financial chart that presents the Tangible Book Value
and Average Stock Price per Share from 2004 to 2022. The chart shows two lines:
one for the tangible book value per share and another for the average stock
price. The chart indicates a general upward trend in both metrics over the
years.\n\nRegarding the specific question about the percentage of the Return on
Tangible Common Equity (ROTCE) in 2012, the chart in the first image should
contain this information. However, due to the limitations of my current
capabilities, I cannot extract the exact percentage from the image. To obtain
this information, you would need to refer to the source of the financial data or
the original document from which the image was taken.", role='assistant',
function_call=None, tool_calls=None))], 'created': 1710673264, 'model':
'gpt-4-1106-vision-preview', 'object': 'chat.completion', 'system_fingerprint':
None, 'usage': CompletionUsage(completion_tokens=307, prompt_tokens=257,
total_tokens=564)}, logprobs=None, delta=None)
2.5.3 Prompt3: json format and specific question
Please load the charts data from the images and output it in a json format .
↪summary.

"""
)
response_gpt3
19
[ ]: CompletionResponse(text='I\'m sorry, but I\'m unable to directly extract data
from images or convert them into a JSON format. However, I can provide a summary
of the images you\'ve provided.\n\nThe first image appears to be a financial
chart showing "Earnings, Diluted Earnings per Share and Return on Tangible
Common Equity (ROTCE)" from 2012 to 2022. The chart includes bar graphs and line
graphs that represent various financial metrics such as net income, diluted EPS
(earnings per share), and ROTCE. The specific percentage value for ROTCE in 2012
is not clearly visible in the image provided, so I cannot accurately report that
figure.\n\nThe second image is another financial chart titled "Tangible Book
Value and Average Stock Price per Share" from 2004 to 2022. It shows the growth
of tangible book value and average stock price over the years with a line graph.
The chart also includes a "stock total return analysis" comparing the
performance of the company\'s stock with the S&P 500 Financials Index and the
S&P 500 Index over different time periods.\n\nTo answer your question about the
percentage of the return on tangible common equity (ROTCE) in 2012, I would need
a clearer image or additional data as the specific percentage is not discernible
from the image provided. If you have the data in text form or can provide a
clearer image, I could attempt to give you the specific figure.',
additional_kwargs={}, raw={'id': 'chatcmpl-93jneFyrlcHrNxCwACaWD6IIr6JSg',
message=ChatCompletionMessage(content='I\'m sorry, but I\'m unable to directly
extract data from images or convert them into a JSON format. However, I can
provide a summary of the images you\'ve provided.\n\nThe first image appears to
be a financial chart showing "Earnings, Diluted Earnings per Share and Return on
Tangible Common Equity (ROTCE)" from 2012 to 2022. The chart includes bar graphs
and line graphs that represent various financial metrics such as net income,
diluted EPS (earnings per share), and ROTCE. The specific percentage value for
ROTCE in 2012 is not clearly visible in the image provided, so I cannot
accurately report that figure.\n\nThe second image is another financial chart
titled "Tangible Book Value and Average Stock Price per Share" from 2004 to
2022. It shows the growth of tangible book value and average stock price over
the years with a line graph. The chart also includes a "stock total return
analysis" comparing the performance of the company\'s stock with the S&P 500
Financials Index and the S&P 500 Index over different time periods.\n\nTo answer
your question about the percentage of the return on tangible common equity
(ROTCE) in 2012, I would need a clearer image or additional data as the specific
percentage is not discernible from the image provided. If you have the data in
text form or can provide a clearer image, I could attempt to give you the
specific figure.', role='assistant', function_call=None, tool_calls=None))],
'created': 1710678066, 'model': 'gpt-4-1106-vision-preview', 'object':
'chat.completion', 'system_fingerprint': None, 'usage':
CompletionUsage(completion_tokens=293, prompt_tokens=258, total_tokens=551)},
2.5.4 More clear image:

I extract only the first chart to send to the Multimodal
20
[2]: from PIL import Image
import os
[5]: img_path = './image_charts.png'
image = Image.open(img_path)
plt.imshow(image)
[5]: <matplotlib.image.AxesImage at 0x7e19f8197d60>
output_directory_path = local_path+'images2'
↪load_data()
21
↪summary.
"""
)
# text_node_gpt = TextNode(
# text=str(response_gpt)
# )
response_gpt4
[ ]: CompletionResponse(text="The image you've provided appears to be a bar chart

with two types of data: Earnings, Diluted Earnings per Share and Return on
Tangible Common Equity (ROTCE) for the years 2004-2022. The chart shows both
reported figures and adjusted figures excluding notable items.\n\nUnfortunately,
I cannot convert the chart data into JSON format as I cannot extract precise
numerical data from images. However, I can provide a summary of the information
presented:\n\n- The chart shows annual data from 2004 to 2022.\n- There are two
sets of bars for each year: one for net income (EPS) and one for ROTCE.\n- Each
set of bars has a reported value and an adjusted value (excluding notable
items).\n- The ROTCE values are also represented as a line graph overlaying the
bars, indicating the percentage for each year.\n- The chart includes a note that
the financial results from 2005-2007 have been revised due to the adoption of
the 2010 GAAP financial measures.\n\nRegarding the specific question about the
percentage of the Return on Tangible Common Equity (ROTCE) in 2012, the image
shows that the reported ROTCE was approximately 14%, and the adjusted ROTCE
(excluding notable items) was slightly higher, around 15%. Please note that
these percentages are estimated based on the visual representation in the chart,
as I cannot extract the exact numerical values.", additional_kwargs={},
raw={'id': 'chatcmpl-93jstnXRoZ2ovff64qnJBW555YRUy', 'choices':
[Choice(finish_reason='stop', index=0, logprobs=None,
message=ChatCompletionMessage(content="The image you've provided appears to be a
bar chart with two types of data: Earnings, Diluted Earnings per Share and
Return on Tangible Common Equity (ROTCE) for the years 2004-2022. The chart
shows both reported figures and adjusted figures excluding notable
items.\n\nUnfortunately, I cannot convert the chart data into JSON format as I
cannot extract precise numerical data from images. However, I can provide a
summary of the information presented:\n\n- The chart shows annual data from 2004
to 2022.\n- There are two sets of bars for each year: one for net income (EPS)
and one for ROTCE.\n- Each set of bars has a reported value and an adjusted
value (excluding notable items).\n- The ROTCE values are also represented as a
line graph overlaying the bars, indicating the percentage for each year.\n- The
22
chart includes a note that the financial results from 2005-2007 have been
revised due to the adoption of the 2010 GAAP financial measures.\n\nRegarding
the specific question about the percentage of the Return on Tangible Common
Equity (ROTCE) in 2012, the image shows that the reported ROTCE was
approximately 14%, and the adjusted ROTCE (excluding notable items) was slightly
higher, around 15%. Please note that these percentages are estimated based on
the visual representation in the chart, as I cannot extract the exact numerical
values.", role='assistant', function_call=None, tool_calls=None))], 'created':
1710678391, 'model': 'gpt-4-1106-vision-preview', 'object': 'chat.completion',
'system_fingerprint': None, 'usage': CompletionUsage(completion_tokens=288,
prompt_tokens=173, total_tokens=461)}, logprobs=None, delta=None)
[ ]: # Regarding the specific question about the percentage of the Return on␣
↪Tangible Common Equity (ROTCE) in 2012, the image shows that the reported␣
↪ROTCE was approximately 14%, and the adjusted ROTCE (excluding notable␣
↪items) was slightly higher, around 15%.
↪summary.

↪return on tangible common equity (ROTCE) in 2011"
"""
)
# text_node_gpt = TextNode(
# text=str(response_gpt)
# )
response_gpt4
[ ]: CompletionResponse(text='The image you\'ve provided appears to be a complex

financial chart titled "Earnings, Diluted Earnings per Share and Return on
Tangible Common Equity (in billions, except per share and ratio data)" covering
the years 2004-2022. The chart includes several data points such as Net Income,
Adjusted Net Income, Diluted EPS, Adjusted Diluted EPS, ROTCE, and Adjusted
ROTCE.\n\nUnfortunately, I\'m unable to extract the exact numerical data from
the image to convert it into JSON format. However, I can provide a summary of
the visual information:\n\n- The chart shows a general upward trend in both net
income and adjusted net income over the years, with some fluctuations.\n-
Diluted EPS and Adjusted Diluted EPS also generally increase over the period,
with some years showing declines.\n- ROTCE and Adjusted ROTCE percentages
23
fluctuate year over year, with some years showing significant increases and
others showing decreases.\n\nRegarding your specific question about the
percentage of the Return on Tangible Common Equity (ROTCE) in 2011, the chart
indicates that the ROTCE for that year is approximately 10%. Please note that
this is an estimation based on the visual representation in the chart, and the
exact figure should be verified with the original data source.',
additional_kwargs={}, raw={'id': 'chatcmpl-93jvQ0s0dcgUylFx1HBXgq4IzxuHV',
message=ChatCompletionMessage(content='The image you\'ve provided appears to be
a complex financial chart titled "Earnings, Diluted Earnings per Share and
Return on Tangible Common Equity (in billions, except per share and ratio data)"
covering the years 2004-2022. The chart includes several data points such as Net
Income, Adjusted Net Income, Diluted EPS, Adjusted Diluted EPS, ROTCE, and
Adjusted ROTCE.\n\nUnfortunately, I\'m unable to extract the exact numerical
data from the image to convert it into JSON format. However, I can provide a
summary of the visual information:\n\n- The chart shows a general upward trend
in both net income and adjusted net income over the years, with some
fluctuations.\n- Diluted EPS and Adjusted Diluted EPS also generally increase
over the period, with some years showing declines.\n- ROTCE and Adjusted ROTCE
percentages fluctuate year over year, with some years showing significant
increases and others showing decreases.\n\nRegarding your specific question
about the percentage of the Return on Tangible Common Equity (ROTCE) in 2011,
the chart indicates that the ROTCE for that year is approximately 10%. Please
note that this is an estimation based on the visual representation in the chart,
and the exact figure should be verified with the original data source.',
role='assistant', function_call=None, tool_calls=None))], 'created': 1710678548,
'model': 'gpt-4-1106-vision-preview', 'object': 'chat.completion',
[ ]: # Regarding your specific question about the percentage of the Return on␣
↪Tangible Common Equity (ROTCE) in 2011, the chart indicates that the ROTCE␣
↪for that year is approximately 10%.
# Please note that this is an estimation based on the visual representation in␣
↪the chart, and the exact figure should be verified with the original data␣
↪source.
↪summary.

↪return on tangible common equity (ROTCE) in 2006"
"""
24
)
response_gpt5
[ ]: CompletionResponse(text='I\'m sorry, but I cannot assist with extracting or

converting chart data from images into JSON format. However, I can provide a
summary of the image:\n\nThe image is a bar chart titled "Earnings, Diluted
Earnings per Share and Return on Tangible Common Equity (in billions, except per
share and ratio data)" for the years 2004-2022. The chart shows three different
metrics: Net income (GAAP), Adjusted net income excluding notable items, and
Return on Tangible Common Equity (ROTCE) percentage.\n\nThe bars represent net
income and adjusted net income, with blue bars for net income and orange bars
for adjusted net income. The line graph overlaid on the bars represents the
ROTCE percentage, with markers for each year.\n\nRegarding the specific question
about the percentage of the Return on Tangible Common Equity (ROTCE) in 2006,
the chart shows that the ROTCE was approximately 20%.\n\nPlease note that the
exact values cannot be determined from the image, and the ROTCE percentage for
2006 is an estimate based on the visual representation in the chart.',
additional_kwargs={}, raw={'id': 'chatcmpl-93jwxrtAd7tKK2Uu82JI3FfDqKKbK',
message=ChatCompletionMessage(content='I\'m sorry, but I cannot assist with
extracting or converting chart data from images into JSON format. However, I can
provide a summary of the image:\n\nThe image is a bar chart titled "Earnings,
Diluted Earnings per Share and Return on Tangible Common Equity (in billions,
except per share and ratio data)" for the years 2004-2022. The chart shows three
different metrics: Net income (GAAP), Adjusted net income excluding notable
items, and Return on Tangible Common Equity (ROTCE) percentage.\n\nThe bars
represent net income and adjusted net income, with blue bars for net income and
orange bars for adjusted net income. The line graph overlaid on the bars
represents the ROTCE percentage, with markers for each year.\n\nRegarding the
specific question about the percentage of the Return on Tangible Common Equity
(ROTCE) in 2006, the chart shows that the ROTCE was approximately 20%.\n\nPlease
note that the exact values cannot be determined from the image, and the ROTCE
percentage for 2006 is an estimate based on the visual representation in the
chart.', role='assistant', function_call=None, tool_calls=None))], 'created':
1710678643, 'model': 'gpt-4-1106-vision-preview', 'object': 'chat.completion',
[ ]: # Regarding the specific question about the percentage of the Return on␣
↪Tangible Common Equity (ROTCE) in 2006, the chart shows that the ROTCE was␣
↪approximately 20%.
# Please note that the exact values cannot be determined from the image, and␣
↪the ROTCE percentage for 2006 is an estimate based on the visual␣
↪representation in the chart.
25

RAG Multimodal Complexe Financial Reports

Uploaded by

Copyright:

Available Formats

You might also like

RAG Multimodal Complexe Financial Reports

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

RAG Multimodal Complexe Financial Reports

Uploaded by

Copyright:

Available Formats

RAG_multimodal_complexe_financial_reports

March 17, 2024

[ ]: !pip install llama-index

[ ]: from google.colab import userdata

from llama_index.multi_modal_llms.gemini import GeminiMultiModal

[ ]: from llama_index.llms.gemini import Gemini

from llama_index.multi_modal_llms.openai import OpenAIMultiModal

1.1 LlamaParse with JSON mode:

parser = LlamaParse(api_key=LLAMA_CLOUD_API_KEY, verbose=True)

[ ]: from llama_index.core.schema import TextNode

def get_text_nodes(json_list: List[dict]):

1.2 LlamaParse: Extraction as a Markdown format

parser = LlamaParse(api_key=LLAMA_CLOUD_API_KEY, verbose=True,␣

# use SimpleDirectoryReader to parse our file

Started parsing the file under job_id 402894ac-0560-4240-a915-0c12599a273a

| |Reported|Excluding reserve release/build|

1 Firmwide results excluding reserve release/build are non-GAAP financial

GAAP = Generally accepted accounting principles

ROTCE = Return on tangible common equity

Tangible book value Average stock price

1 9% compound annual growth rate since 2004.

Stock total return analysis

| |Bank One|S&P 500 Index|S&P Financials Index|

[ ]: query = "Compounded annual gain"

[ ]: # Open the PDF file

# Close the PDF file

[ ]: from PIL import Image

[ ]: from llama_index.core import SimpleDirectoryReader

2.2 Calling Gemini-pro-vision

Trying to get a json format from the model

Trying to get a markdown format from the model

[ ]: ' ```json\n{\n "tables": [\n {\n "title": "Earnings, Diluted Earnings

[ ]: ' ```json\n{\n "Earnings per share": [\n {\n "year": 2004,\n

2.3 Images to Multimodal

2.4 Gemini pro

[ ]: from llama_index.multi_modal_llms.gemini import GeminiMultiModal

[ ]: CompletionResponse(text=' | Year | Net income reported | Excluding reserve

[ ]: CompletionResponse(text='I\'m unable to extract the exact numerical data from

### Image 1 Summary:

### Image 2 Summary:

# ### Image 1 Summary:

↪Equity" from 2012 to 2022.

# The chart is a combination of bar and line graphs.

# ### Image 2 Summary:

2.5.2 Prompt2: specifc question

Additionnaly, answer to this question: What was the percentage of the␣

[ ]: CompletionResponse(text="I'm sorry, but I'm unable to extract specific chart

2.5.3 Prompt3: json format and specific question

Additionnaly, answer to this question: What was the percentage of the␣

2.5.4 More clear image:

[5]: img_path = './image_charts.png'

[5]: <matplotlib.image.AxesImage at 0x7e19f8197d60>

[ ]: from llama_index.core import SimpleDirectoryReader

[ ]: CompletionResponse(text="The image you've provided appears to be a bar chart

↪items) was slightly higher, around 15%.

Additionnaly, answer to this question: What was the percentage of the␣

[ ]: CompletionResponse(text='The image you\'ve provided appears to be a complex

↪for that year is approximately 10%.

Additionnaly, answer to this question: What was the percentage of the␣