Beyond text: images, voice, documents and video

In this lesson you will use AI with four kinds of input besides typed text: a photo, your voice, a PDF and a video. By the end you will have explained a textbook diagram from a photo, practised a short spoken interview, asked questions about a PDF with page-level citations, and summarised a YouTube lecture.

What you need: a phone with the free Gemini or ChatGPT app (or a laptop browser), a Google account for NotebookLM, and one PDF you are allowed to share, such as your own class notes or a public government report. No coding.

A multimodal model can take in more than one kind of input: text, images, audio, documents and sometimes video. Some can also produce images or speak their answers aloud.

Under the hood, everything becomes tokens. A photo is cut into small patches, audio into short slices, and the model processes them the same way it processes words (covered in *Tokens, context windows and why chats forget*). That is why a long video or a 300-page PDF can fill up the context window just like a long chat.

I've attached a photo of a diagram from my [subject] textbook about [topic].

1. First, list every label and arrow you can read in the image, exactly as written. Say "unclear" for anything you can't read.
2. Then explain what the diagram shows, step by step, in simple English for a [year/level] student.
3. End with 2 questions I could be asked about this diagram in an exam.

What to expect: Asking it to transcribe the labels first is the key step: you can check that list against the photo in seconds, and if the transcription is wrong, the explanation will be wrong too.

This screenshot shows an error I got while [what you were doing, e.g. installing Python on Windows 11].

Read the exact error text from the screenshot first and quote it. Then explain in plain English what it means, the most likely cause, and the steps to fix it. If the screenshot doesn't show enough, tell me what else to capture.

What to expect: Use a screenshot (Win+Shift+S on Windows, Power+Volume Down on most Android phones), not a photo of your screen. Screenshots are sharp, so the text is read correctly.

Voice mode lets you talk to the AI and hear it answer, like a phone call. Gemini Live and ChatGPT's voice mode are both available in their free apps, with usage limits. You can interrupt mid-sentence, which makes it feel natural.

Voice is most useful when the skill you are practising is itself spoken: answering interview questions, explaining a concept aloud, or improving spoken English. Many tools understand Hindi and other Indian languages, so you can also ask a question in the language you think in and get the answer in English.

You are an interviewer for a [role, e.g. fresher software developer] position at [type of company, e.g. an Indian IT services company]. Ask me one question at a time, wait for my full answer, and don't give feedback until I say "feedback". Start with "Tell me about yourself", then ask two questions about [topic, e.g. my final-year project / Java / SQL]. When I say "feedback", tell me for each answer: what was clear, what was missing, and one filler habit to drop.

What to expect: A good session keeps to one question at a time and gives specific feedback ("you didn't say what your role in the project was"), not general praise.

You can attach a PDF to any major chatbot and ask questions about it. For study and research, NotebookLM (from Google, free with a Google account) is often the better choice: it answers only from the sources you add, and every claim comes with a numbered citation you can click to see the exact passage.

Two things affect quality. First, a scanned PDF is just a picture of pages; text PDFs (where you can select words) are read far more reliably. Second, very long documents may not fit or may be summarised loosely. Techniques for long files are covered in *Working with long documents and files* in Prompt Engineering.

Using only the uploaded source(s), answer: [your question].

Rules:
- Quote the exact sentence(s) your answer is based on.
- If the source doesn't answer it, say "Not in the source" instead of guessing.
- Keep the answer under [150] words.

What to expect: The "Not in the source" rule matters. Without it, a chatbot may fill gaps from general knowledge and you won't know which parts came from your document.

In the Gemini app you can paste a public YouTube link and ask for a summary, the key points or where a topic is explained. NotebookLM can take a YouTube link as a source too. In both cases the tool mainly works from the video's transcript (the spoken words), so it can miss what is only shown on screen, like code on a slide or a diagram on a whiteboard.

You can also upload an audio recording, such as your own voice note. Only record lectures or meetings when you have permission, and never upload recordings of other people's private conversations.

Here is a lecture video: [YouTube link]

1. Summarise it in 8 bullet points for a [level] student.
2. List the main topics with approximate timestamps, so I can jump to each one.
3. List any definitions or formulas the speaker states, quoted as closely as possible.
4. Tell me what parts seem to rely on visuals (slides, board work) that you may not be able to see.

What to expect: Check two or three timestamps by jumping to them. If they're wrong, treat the whole summary with caution. Point 4 reminds you what to watch yourself.

Gemini, ChatGPT, Meta AI and Microsoft Copilot can create images from a description. They are good for illustrations, posters and slide visuals. They are weaker at exact text inside images, correct diagrams (a labelled heart or a circuit), and the same character across many images.

How to describe images well is covered in *Prompting image and media generators* in Prompt Engineering. Two rules for now: don't create realistic images of real people without their consent, and don't pass off a generated image as a real photo. Some tools add invisible watermarks (Google uses one called SynthID) so that AI images can be detected.