Skip to content
Introduction
Few days ago, Google presented their own multimodal-LLM named as “Gemini”.
Also there was article named “How to Integrate google’s gemini AI model into R” that tells us how to use gemini API in R brieflly.
Thanks to Deepanshu Bhalla (writer of above article), I’ve many inspirations and made some research to utilize Gemini API more. And I’m glad to share the results with you.
In this article, I want to highlight to How to use gemini with R and Shiny via R package for Gemini API
(You can see result and contribute in github repository: gemini.r)
Gemini API
As today (23.12.26), Gemini API is mainly consisted with 4 things. you can see more details in official docs.
1. Gemini Pro: Is get Text and returns Text
2. Gemini Pro Vision: Is get Text and Image and returns Text
3. Gemini Pro Multi-turn: Just chat
4. Embedding: for NLP
and I’ll use 1 & 2.
You can get API keys in Google AI Studio
However, offical docs doesn’t describe for how to use Gemini API in R. (How sad)
But we can handle it as “REST API” ( I’ll explain it later)
Shiny application
I made very brief concept of Shiny application that uses Gemini API for get Image and Text (maybe “Explain this picture”) and returns Answer from Gemini
(Number is expected user flow)
This UI, is consisted 5 components.
1. fileInput for upload image
2. imageOutput for show uploaded Image
3. textInput for prompt
4. actionButton for send API to Gemini
5. textOutput for show return value from Gemini
And this is result of shiny and R code (Again, you can see it in github repository)
—
library(shiny)
library(gemini.R)
ui <- fluidPage(
sidebarLayout(
NULL,
mainPanel(
fileInput(
inputId = “file”,
label = “Choose file to upload”,
),
div(
style = ‘border: solid 1px blue;’,
imageOutput(outputId = “image1”),
),
textInput(
inputId = “prompt”,
label = “Prompt”,
placeholder = “Enter Prompts Here”
),
actionButton(“goButton”, “Ask to gemini”),
div(
style = ‘border: solid 1px blue; min-height: 100px;’, textOutput(“text1”)
)
)
)
)
server <- function(input, output) {
observeEvent(input$file, {
path <- input$file$datapath
output$image1 <- renderImage({
list( src = path )
}, deleteFile = FALSE) })
observeEvent(input$goButton, {
output$text1 <- renderText({
gemini_image(input$prompt, input$file$datapath)
})
})
}
shinyApp(ui = ui, server = server)
—
gemini.R package
I think you may think “What is gemini_image function?”
It is function to send API to Gemini server and return result.
and it consisted with 3 main part.
1. Model query
2. API key
3. Content
I used gemini_image function in example. but I’ll gemini function first (which is function to send text and get text)
Gemini’s API example usage is looks like below. (for REST API)
Which can be transformed like below in R
Also, gemini API key must set before use with “Sys.setenv” function
Anyway, I think you should note, body for API is mainly consisted with list.
Similarly, gemini_image function for Gemini Pro Vision API looks like below
is
Note that, image must encoded as base64 using base64encode function and provided as separated list.
Example
So with Shiny application and gemini.r package.
You now can run example application to ask image to Gemini.
Summary
I made very basic R package “gemini.R” to use Gemini API.
Which provides 2 function: gemini and gemini_image.
And still there’s many possiblity for develop this package.
like feature to Chat like bard or provide NLP Embedding
and finally, I want to hear feedback or contribution from you. (Really)
Thanks.
* P.S, I think just using bard / chatGPT / copilot is much better for personal usage. (unless you don’t want to provide AI service via R)
- The DN Unlimited Conference will take place online for the first time this year
- More than 100 speakers from the fields of AI, machine learning, data science, and technology for social impact, including from The New York Times, IBM, Bayer, and Alibaba Cloud
- Fully remote networking opportunities via a virtual hub
Europe’s largest data science community launches new digital platform for this year’s conference
The
Data Natives Conference, Europe’s biggest data science gathering, will take place virtually and invite data scientists, entrepreneurs, corporates, academia, and business innovation leaders to connect on November 18-20, 2020. The conference’s mission is to connect data experts, inspire them, and let people become part of the equation again. With its digital networking platform, DN Unlimited expects to reach a new record high with 5000+ participants. Visitors can expect keynotes and panels from the industry experts and a unique opportunity to start on new collaborations during networking and matchmaking sessions.
In 2019, the sold-out Data Natives conference gathered over 3000 data, technology professionals and decision-makers from over 30 countries, including 29 sponsors, 45 community and media partners, and 176 speakers.The narrative of DN Unlimited Conference 2020 focuses on assisting the digital transformation of businesses, governments, and communities by offering a fresh perspective on data technologies – from empowering organizations to revamp their business models to shedding light on social inequalities and challenges like Climate Change and Healthcare accessibility.
Data science, new business models and the future of our society
In spring 2020, the Data Natives community of 80.000 data scientists mobilised to tackle the challenges brought by the pandemic – from the shortage of medical equipment to remote care – in a series of Hackcorona and EUvsVirus hackathons. Through the collaboration of governments such as the Greek Ministry for Digital Governance, institutions such as the Charité and experts from all over Europe, over 80 data-driven solutions have been developed. DN Unlimited conference will continue to facilitate similar cooperation.
The current crisis demonstrates that only through collaboration, businesses can thrive.
While social isolation may be limiting traditional networking opportunities, we are more equipped than ever before to make connections online. “…The ability to connect to people and information instantly is so common now. It’s just the beginning of an era of even more profound transformation. We’re living in a time of monumental change. And as the cloud becomes ambiguous, it’s literally rewriting entire industries” – says Gretchen O’Hara, Microsoft VP; DN Unlimited & Humanaize Open Forum speaker.
The crisis has called for a digital realignment from both companies and institutions. Elena Poughia, the Founder of Data Natives, perceives the transformation as follows:
“It’s not about deploying new spaces via data or technology – it’s about amplifying human strengths. That’s why we need to continue to connect with each other to pivot and co-create the solutions to the challenges we’re facing. These connections will help us move forward.”
The DN Unlimited Conference will bring together data & technology leaders from across the globe – Christopher Wiggins (Chief Data Scientist, The New York Times), Lubomila Jordanova (CEO & Founder, Plan A), Angeli Moeller (Bayer AG, Head Global Data Assets), Jessica Graves (Founder & Chief Data Officer, Sefleuria) and many more will take on the virtual stages to talk about the growing urge for global data literacy, resources for improving social inequality and building a data culture for agile business development.
On stage among others:
- Erika Cheung, Executive Director, Ethics in Entrepreneurship
- Cory Doctorow, Science Fiction Author, Activist, and Journalist.
- Whurley (William Hurley), Eisenhower Fellow, a Senior Member of the Institute of Electrical and Electronics Engineers (IEEE)
- Alistair Croll, Founder, Solve for Interesting
- Clare Jones, Chief Commercial Officer, what3words
- Mark Turrell, Orcasci, CEO
- And many more …
