I Tried to Construct Picture Captioning App With OpenAI Codex CLI

Abstract

Codex CLI is a terminal-based AI assistant that permits you to work together with OpenAI fashions to learn, modify, and run code utilizing pure language.
Runs domestically for privateness, helps multimodal inputs (textual content + pictures), and integrates seamlessly along with your shell for a easy workflow.
Helps bug fixing, refactoring, studying codebases, and prototyping with adjustable management through Counsel, Auto Edit, and Full Auto modes.
Simple to put in with Node.js and an OpenAI API key, and works finest on Unix-based programs or Home windows through WSL2.

OpenAI Codex CLI is an open‑supply command-line device that brings the facility of OpenAI’s newest reasoning fashions on to your terminal. Consider it as a light-weight AI coding assistant that lives in your shell: it could possibly learn your code, modify recordsdata, and even execute instructions in your mission surroundings. This implies you possibly can ask it to construct options, repair bugs, or clarify unfamiliar code with out leaving your growth workflow. Briefly, it’s chat-driven growth – you work together with Codex in pure language and it responds with code edits or command outcomes, successfully supplying you with ChatGPT-level reasoning plus the flexibility to run code and see outcomes in real-time. Sounds intriguing, proper? Additional on this article, I’ll let you know about the way to entry it and use to your queries.

Key Options of OpenAI Codex CLI

OpenAI Codex CLI comes with a number of highly effective options that make it a useful companion for builders. One of many greatest benefits of Codex CLI is that it runs solely in your native machine. Your supply code and recordsdata keep in your surroundings and aren’t uploaded wholesale to a cloud service. Solely your prompts and high-level context (like summarized diffs or related snippets) are despatched to the OpenAI API for producing responses. As a result of the CLI is open-source and works domestically, it provides you privateness and management by design – your workflow and code stay non-public. This makes Codex CLI particularly interesting for codebases which you could’t or don’t wish to share, whereas nonetheless leveraging highly effective AI help.

By integrating instantly into the terminal, Codex CLI suits naturally right into a developer’s day-to-day work. You’ll be able to chat with the AI assistant proper subsequent to your git instructions, textual content editor, and construct instruments, which suggests much less context-switching in comparison with utilizing a separate chat interface. The device is designed for fast iteration: ask a query or give an instruction, let it suggest or apply a change, run the code, and repeat – multi functional place.

Listed here are a few of the highlights:

1. Zero-Setup Set up

Codex CLI is extraordinarily straightforward to get working. All you want is Node.js and an OpenAI API key – a single command like npm set up -g @openai/codex installs the CLI globally, with no different setup required. There’s no advanced configuration or surroundings fiddling; carry your API key and it “simply works”. (You’ll be able to even replace to the most recent model at any time with a easy codex –improve command.)

2. Terminal-Native Design

Codex runs solely in your terminal, so it looks like a pure extension of your shell surroundings. You’ll be able to invoke it out of your mission listing and have it work together along with your native recordsdata and instruments. This terminal-native strategy means you don’t have to change to a browser or GUI – excellent for sustaining stream and context whereas coding. The CLI gives an interactive chat-like interface in textual content, so that you see the AI’s responses (like code diffs or command outputs) proper within the console.

3. Multimodal Inputs

Not like plain text-only instruments, Codex CLI accepts multimodal inputs – you possibly can go not simply textual content prompts, but additionally pictures resembling screenshots or diagrams to information the assistant. For instance, you would drag a screenshot of an error message or a UI sketch into the terminal, and Codex can interpret it and act on it. This can be a distinctive functionality that lets the AI use visible info to generate or edit code accordingly. Below the hood, it makes use of vision-enabled fashions to grasp pictures, enabling use circumstances like debugging from a screenshot of a stack hint or constructing a format from a wireframe.

4. Wealthy Approvals Workflow

Codex CLI provides you fine-grained management over what it could possibly do autonomously via a wealthy approval system. You’ll be able to select between three modes (Counsel, Auto Edit, Full Auto) that decide whether or not the AI’s proposed code modifications or instructions are auto-executed or require your affirmation. This versatile workflow enables you to resolve how hands-on you wish to be: you can begin conservatively (guide approvals for every part) and dial as much as full automation for repetitive duties. We’ll dive deeper into these modes within the subsequent part, however the important thing level is that Codex gained’t make modifications you’re uncomfortable with – you’re all the time in control of approvals.

5. Native Execution and Privateness

All code execution and file enhancing occurs on your machine, inside your mission’s surroundings. Except for the mannequin queries, nothing is shipped out – the CLI doesn’t add your codebase to OpenAI. This implies you keep full privateness. You’ll be able to safely use Codex CLI on proprietary or delicate code figuring out that the device isn’t retaining or sharing your knowledge. Even when utilizing essentially the most autonomous mode, Codex runs in a sandboxed surroundings with no community entry, making certain any actions it takes keep native to your system. Briefly, you get the advantages of an AI pair programmer with out giving up privateness or safety.

Codex CLI Modes that You Should Know

Github Hyperlink: openai/codex

A standout function of Codex CLI is its approval workflow – basically, you resolve how a lot freedom the AI has to make modifications or run instructions. There are three approval modes: Counsel, Auto Edit, and Full Auto. Every mode strikes a unique stability between automation and consumer oversight, so you possibly can decide what suits your consolation degree for the duty at hand. Right here’s an outline of how they work:

1. Counsel Mode (Default)

That is essentially the most conservative mode, very best for whenever you wish to rigorously assessment every part. The AI can learn your mission recordsdata and recommend code edits or terminal instructions, but it surely gained’t apply modifications or execute something with out your express approval . Primarily, Codex will work together with you want an skilled advisor: it would suggest a patch diff for a bug repair or present a shell command to run exams, after which ask to your affirmation. Use Counsel mode for protected exploration – e.g. studying a brand new codebase or doing a code assessment – the place you wish to see suggestions however apply them manually .

2. Auto Edit Mode

In Auto Edit, Codex is allowed to routinely apply code modifications (it could possibly edit/write to recordsdata by itself) however nonetheless should ask earlier than working any shell instructions . This mode is nice for duties like refactoring or making repetitive edits throughout a codebase. You get the effectivity of the AI instantly modifying code for you, whereas retaining a checkpoint of management earlier than any program execution. For instance, Codex may rewrite a operate in a number of recordsdata and save the modifications instantly, but when it desires to run your check suite or begin the dev server, it’s going to pause and ask to your go-ahead. Auto Edit mode is a stability: quicker coding iterations, but you continue to supervise side-effects like instructions .

3. Full Auto Mode

Full Auto provides the AI essentially the most autonomy. Codex can learn and write recordsdata and in addition execute shell instructions by itself with out stopping for approval . On this mode, it turns into a really automated agent – you would ask it to carry out a fancy job after which sit again whereas it really works via the steps. To maintain issues protected, Full Auto runs in a restricted sandbox: all instructions are executed with community entry disabled and scoped to your mission listing (it could possibly’t wander outdoors or entry the web). This mode is right for longer duties the place you belief the AI to iterate, for example, fixing a damaged construct or prototyping a brand new function when you take a brief break . After all, it is best to use Full Auto with warning – it’s highly effective, however you’ll wish to make sure you’ve backed up or version-controlled your code (the CLI will truly warn you in the event you’re not in a git repo when beginning Auto Edit or Full Auto) .

Comparability of Modes

The variations between the three modes are summarized within the desk under, together with typical use circumstances for every:

Mode	What the Agent Can Do	When to Use (Use Circumstances)
Counsel (default)	– Learn any recordsdata in your repo – Suggest edits and shell instructions (requires your approval to use/execute)	Protected exploration of codebases, code opinions, studying a brand new mission’s construction the place you need full management over modifications .
Auto Edit	– Learn and modify recordsdata (apply edits routinely) – Suggest shell instructions (execution nonetheless requires approval)	Refactoring code or making bulk edits whereas maintaining a tally of negative effects. Nice for repetitive modifications the place guide file enhancing is tedious however you continue to wish to approve any instructions .
Full Auto	– Learn, write, and execute instructions autonomously (all actions auto-approved) – Runs in sandbox (no community, confined to mission listing)	Massive or time-consuming duties like fixing all exams in a damaged construct or scaffolding a brand new app from scratch. Helpful whenever you wish to delegate execution solely to the AI (e.g. fast prototyping) .

In apply, you possibly can choose the mode that is smart to your state of affairs. By default, in the event you simply run codex it begins in Counsel mode. To explicitly select a mode, you possibly can launch the CLI with a flag: for instance, use –auto-edit or –full-auto to start out in these modes . There’s additionally an interactive command (/mode) to toggle modes throughout a session . This fashion, you may start in Counsel mode to see what Codex plans to do, then swap to Auto Edit when you’re snug with its strategies, and perhaps kick into Full Auto for the ultimate stretch of a job. The essential factor is that you management the extent of autonomy always.

System Necessities for Codex CLI

Earlier than putting in Codex CLI, make sure that your growth surroundings meets the minimal necessities. The device is cross-platform, however at the moment works finest on Unix-like programs. Listed here are the minimal and really helpful specs:

Requirement	Minimal	Beneficial
Working System	macOS 12+ or Ubuntu 20.04+/Debian 10+ (Linux)； Home windows 11 through WSL2	Newest OS updates (newest macOS or LTS Linux launch; Home windows with newest WSL2) for finest compatibility.
Node.js	22 (or newer)	Newest LTS model of Node.js (>= 22) for stability.
Git (non-compulsory)	2.23+ (if utilizing model management options)	Latest Git out there (non-compulsory, however really helpful for full performance like PR helpers).
Reminiscence (RAM)	4 GB minimal	8 GB or extra (for smoother efficiency on massive duties).

Codex CLI has been examined on macOS and Linux. Home windows customers can run it through WSL2 (Home windows Subsystem for Linux) since native Home windows help continues to be experimental . You’ll additionally want an OpenAI API key (out of your OpenAI account) to authenticate the CLI – we’ll cowl that subsequent. Except for these, no different particular {hardware} is required; in the event you can run trendy Node.js, you’re probably good to go.

Word: It’s really helpful to have your mission underneath supply management (git) when utilizing Codex CLI, particularly for Auto modes. Whereas Git isn’t strictly required to run the CLI, having model management will let you simply assessment modifications and rollback if wanted. The truth is, Codex will remind you with a warning in the event you attempt to use Auto Edit or Full Auto in a listing that’s not a git repo

Tips on how to Use OpenAI Codex CLI?

Step 1: Set up Node.js

Obtain Node.js v22+ from nodejs.org.
Set up utilizing default settings.
Confirm set up:

bash node --version  # Ought to present v22+ npm --version   # Ought to present v10

Step 2: Set up Codex CLI

bash npm set up -g @openai/codex

Troubleshooting: In case you see permission denied errors:
- Home windows: Run PowerShell as Administrator.
- Linux/macOS: Use sudo npm set up -g @openai/codex (not really helpful; repair npm permissions as an alternative).

Step 3: Set OpenAI API Key

For PowerShell (Home windows):

Powershell

$env:OPENAI_API_KEY = "your-api-key-here"

To make it everlasting:

Powershell

setx OPENAI_API_KEY "your-api-key-here"

For Git Bash/MINGW64:

bash export OPENAI_API_KEY="your-api-key-here"

To make it everlasting, add to ~/.bash_profile:

bash nano ~/.bash_profile  # Add "export OPENAI_API_KEY=..." supply ~/.bash_profile

Step 4: Repair “sh.exe” Errors (Home windows Solely)

Set up Git for Home windows from git-scm.com.
Throughout set up:
- Choose “Use Git and Unix instruments within the Command Immediate”.
- Allow “Allow symbolic hyperlinks”.
Restart your terminal.

Step 5: Run Codex

Interactive Mode

Run interactively:

Codex

Fingers-on OpenAI Codex CLI to Construct Recreation and Picture Captioning App

Process 1: Fundamental Immediate Execution

I began with a easy job—asking Codex to jot down 2–3 sentences about myself. The CLI responded rapidly and precisely, producing coherent, grammatically sound output in simply seconds. It demonstrated robust immediate understanding and fluency, even with minimal enter.

Process 2: Picture Captioning App with OpenAI Mannequin

Subsequent, I attempted constructing a extra advanced software: a picture captioning device the place customers add a picture and obtain a descriptive caption generated by an OpenAI mannequin. Whereas Codex supplied a good place to begin, the code was outdated—referencing deprecated code and lacking key elements for file dealing with and mannequin integration. I needed to step in and replace the code myself. (I’ve included a screenshot for reference.) This highlighted a limitation: for newer or less-documented APIs, Codex may fall again on older patterns or incomplete implementations.

Error with Codex CLI

Process 3: Tetris Recreation with Python and Pygame

Output

For the ultimate job, I requested Codex to construct a Tetris sport utilizing Python and Pygame. This time, it nailed it. The code was well-structured, totally practical, and required no main edits. The sport ran easily and included all of the core mechanics—block motion, rotation, line clearing, and scoring. A strong demonstration of Codex’s skill to deal with interactive, graphics-based initiatives when working with well-established libraries like Pygame.

Use Circumstances for Codex CLI

Codex CLI can supercharge your growth workflow throughout a number of widespread duties:

Bug Fixing: While you hit a bug or failing check, use Counsel mode to ask issues like “Why is the login operate throwing an error?” Codex analyzes the code, spots points (like a unsuitable variable or lacking verify), and suggests fixes. You assessment and approve the patch. For trickier points, Full Auto mode lets Codex repair a number of failures by iteratively working exams and making use of modifications. You continue to confirm the outcomes, but it surely handles the heavy lifting.
Code Refactoring: Refactoring throughout recordsdata—like switching from callbacks to async/await—may be tedious. In Auto Edit mode, Codex can apply constant modifications all through your codebase. For instance, say “Refactor the API routes to async/await,” and it’ll deal with the file edits, pausing provided that wanted. You supervise the modifications through diffs, letting Codex do the grunt work when you oversee high quality.
Studying a New Codebase: Simply cloned a repo? Use Counsel mode to ask, “What does the Scheduler class do?” or “How does authentication work?” Codex reads the code and explains in plain language, serving to you navigate unfamiliar initiatives rapidly. You’ll be able to request summaries, perceive module duties, and discover performance with out making modifications.
Prototyping and Scaffolding: Need to kickstart a brand new mission or function? Full Auto mode can generate code and set every part up. Ask it to “Create a easy TODO net app in Flask,” and it’ll generate recordsdata, set up dependencies, and run the app—routinely. For brand new options like “Add CSV export to this CLI device,” Codex writes and integrates the code, supplying you with a working baseline to construct on.

Codex CLI acts like an AI pair-programmer—serving to with every part from mundane edits to advanced automation. You management how hands-on or autonomous it’s, relying on the duty.

Conclusion

With the OpenAI Codex CLI, builders acquire a pleasant AI associate proper within the terminal – one that may motive about code and deal with the mechanics of enhancing and working it. I’ve lined what Codex CLI is and the way it works, from its zero-effort set up to the intelligent approval modes that hold you in management. You’ve seen the way to get began and run some fundamental instructions, and the way it may help in real-world use circumstances like fixing bugs, refactoring, studying codebases, and prototyping new concepts. In essence, Codex CLI brings the ChatGPT expertise into your growth surroundings, turning pure language directions into working code, all when you stay in cost. It’s an thrilling device that embodies the way forward for AI-assisted software program growth: quick, versatile, and constructed with developer empowerment in thoughts. Give it a attempt in your subsequent mission!

Hello, I’m Pankaj Singh Negi – Senior Content material Editor | Captivated with storytelling and crafting compelling narratives that rework concepts into impactful content material. I like studying about know-how revolutionizing our life-style.