# Reading agent code as if the input were hostile > Run a static analyzer over an agent-written search function, find the string-built query it reports and the swallowed error it misses, and brief the agent for a parameterized fix with a test. Page: https://schubergphilis.github.io/ai-training/coding-with-agents/security-review/ License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/ A coding agent added a search box to a small notes app. The brief was short: a user types part of a title and sees their own notes that match. The agent wrote a search function and a small admin check, and the search works when you try it. In this lesson we read that change the way an attacker would. A static analyzer finds part of what is wrong with it, and we read the code for the rest. Then we brief the agent for a fix, with a test that proves it. Brilliant's *Coding with AI* skills map lists security and adversarial thinking as a skill area of its own (Security and adversarial thinking, Brilliant, Coding with AI skills map). The question this lesson practices is the adversarial one: what happens when the input is hostile? ## The agent's change The fixture is in `site/examples/coding-with-agents/security-review/` in your clone of the course repository. From the root of your clone, copy it somewhere outside the clone, so that nothing you run touches the repository. If `~/security-review` exists from an earlier try, delete it first, or `cp` puts the new copy inside the old one. ```sh cp -R site/examples/coding-with-agents/security-review ~/security-review cd ~/security-review ``` `notes_search.py` is the agent's change. The route that calls `search_notes` passes the logged-in user's name as `owner` and the text from the search box as `term`. This is the whole function, with the admin check the agent added in the same commit: ```python ADMIN_PASSWORD = "notes-admin-2026" def is_admin(password: str) -> bool: """Lets the admin page in when the password matches.""" return password == ADMIN_PASSWORD def search_notes(conn: sqlite3.Connection, owner: str, term: str) -> "list[str]": """Returns the titles of the owner's notes whose title contains term.""" query = ( f"SELECT title FROM notes WHERE owner = '{owner}' AND title LIKE '%{term}%' ORDER BY title" ) try: rows = conn.execute(query).fetchall() except sqlite3.Error: return [] return [row[0] for row in rows] ``` It reads cleanly and it does what the brief asked for. Keep it open while you read the next section. ## The mistakes agents copy A model learns to write code from a large amount of public code, and public code contains bugs, including security bugs. In a study from 2021, Pearce and colleagues gave GitHub Copilot 89 coding scenarios built around high-risk weakness types from the Top 25 list that the MITRE Corporation publishes. They judged about 40 percent of the 1,689 programs it produced to be vulnerable. Their explanation is that a model trained on code that nobody checked has learned the exploitable patterns in it (Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions, 2022 IEEE Symposium on Security and Privacy (SP 2022), 754-768). Models have changed since that study, and they're still trained on public code. The list below is this course's own, of mistakes that appear in a lot of that code: - **Unvalidated input.** A value from a form, a file or an application programming interface (API) is used as if it were trusted: a quantity that can be negative, a file name that can contain `../`. - **String-built queries and commands.** Input is pasted into an SQL (Structured Query Language) query or a shell command, so the input can change what the query or the command does. - **Secrets in code.** A password, token or key is written into the source, where every copy of the repository has it. - **Permissive defaults.** A server listens on every network interface, a debug mode stays on, or a file is created readable by everyone. - **Disabled checks.** Certificate checking is turned off to make an error go away, or a test is skipped because it failed. - **Error handling that hides failures.** An exception is caught and replaced with an empty result. A failure then looks the same as "nothing found". Each of these makes the code shorter or makes an error go away. An agent that is told to get the task done uses them for the same reason a hurried person does. #### Checkpoint: Which lines stop the review? Each line comes from a different agent-written change. Which lines show one of the classic mistakes? Select exactly 3. - `requests.get(url, verify=False)` - `cursor.execute("SELECT title FROM notes WHERE owner = ?", (owner,))` - `os.system("convert " + name + " thumb.png")` - `token = os.environ["NOTES_API_TOKEN"]` - `app.run(host="0.0.0.0", debug=True)` - `except sqlite3.Error: log.exception("search failed"); raise` ## What the analyzer finds A static analyzer reads source code without running it and reports patterns that are often mistakes. For Python, Bandit is a free, Apache-2.0 licensed analyzer that looks for common security problems (Bandit documentation, PyCQA). In the exercise at the end of this lesson, you install Bandit and run it on your own copy of the fixture. The recorded run below is for reading now, and for readers who skip the install. On September 27, 2026, Bandit 1.8.6 running on Python 3.9.25 printed this for `notes_search.py` in a fresh copy of the fixture, with the start time left out: ```text Test results: >> Issue: [B105:hardcoded_password_string] Possible hardcoded password: 'notes-admin-2026' Severity: Low Confidence: Medium CWE: CWE-259 (https://cwe.mitre.org/data/definitions/259.html) More Info: https://bandit.readthedocs.io/en/1.8.6/plugins/b105_hardcoded_password_string.html Location: ./notes_search.py:11:17 10 11 ADMIN_PASSWORD = "notes-admin-2026" 12 -------------------------------------------------- >> Issue: [B608:hardcoded_sql_expressions] Possible SQL injection vector through string-based query construction. Severity: Medium Confidence: Low CWE: CWE-89 (https://cwe.mitre.org/data/definitions/89.html) More Info: https://bandit.readthedocs.io/en/1.8.6/plugins/b608_hardcoded_sql_expressions.html Location: ./notes_search.py:22:8 21 query = ( 22 f"SELECT title FROM notes WHERE owner = '{owner}' AND title LIKE '%{term}%' ORDER BY title" 23 ) -------------------------------------------------- ``` The run also printed a summary with the line count and the issues per severity, and it exited with status 1 because it found issues. The column of the B608 finding depends on the Python version: 3.9, 3.10 and 3.11 printed `22:8`, and 3.12 and 3.13 printed `22:10`. Bandit found two of the classic mistakes. B105 is a password in the source, which Bandit spots because the variable name contains `password`. B608 is the query built from strings. Bandit rates its confidence in that finding as Low. A pattern check can't see where `owner` and `term` come from. You can, and here `term` comes from a user, so the finding is real. Bandit didn't report the `except sqlite3.Error: return []`. One of its checks, B110, looks for an `except` block whose only statement is `pass` (Bandit documentation, PyCQA). This block returns an empty list, which hides the error just as well and matches no rule. An analyzer finds patterns that someone wrote a rule for. What the code means for this app, such as whether an empty list means that nothing matched, is a question only a reader can answer. #### Checkpoint: What did the analyzer miss? Bandit has rules that catch hardcoded passwords, SQL built from strings, and an `except` block whose only statement is `pass`. Which problem in `notes_search.py` does it not report? - The query that is built from the search text with an f-string. - The admin password that is written into the source file. - The except block that turns a database error into an empty list. - The use of the sqlite3 module where an object-relational mapper is expected. #### Pitfall: The search that finds nothing Alice searches for her note "Call O'Brien about the lease" and the app says she has no matching notes. The apostrophe in `O'Brien` ends the quoted string in the SQL too early, and the database rejects the query. The `except` block turns that error into an empty list without logging it, and it would hide an attacker's failed attempts in the same way. Read every `except` in agent code, ask what the user sees when it runs, and let an error that nobody handles reach the caller or the log. ## What if the input is hostile Now ask the question at each place where data enters the function, and where its error goes: - `term` comes from the search box. Anyone who can log in controls all of it. - `owner` comes from the login session. It's trusted only as long as the route takes it from the session, and a later change that reads it from the web address (URL) makes it hostile too. - The database error goes back to the user as an empty list, so the user can't tell a failure from "no match". Take `term` first. Suppose Alice types `%' OR owner != '` into the search box. The f-string pastes it into the query, and the `WHERE` clause becomes `owner = 'alice' AND title LIKE '%%' OR owner != '%'`. `AND` binds more tightly than `OR` in SQL. #### Checkpoint: Predict how many titles come back The database holds four notes, two of Alice's and two of Bob's. How many titles does `search_notes(conn, "alice", "%' OR owner != '")` return? ```python print(len(search_notes(notes_db(), "alice", "%' OR owner != '"))) ``` `hostile.py` runs the search as Alice with three terms: the ordinary one the brief had in mind, the hostile one, and a title with an apostrophe. #### Example: Run the three searches From `~/security-review`, run the script and compare what you see with the output below. ```sh python3 hostile.py ``` Output: ```text 'plan': ['Plan the team offsite'] "%' OR owner != '": ['Bank login reminder', "Call O'Brien about the lease", 'Plan salary review', 'Plan the team offsite'] "O'Brien": [] ``` The second line shows Bob's notes to Alice, including one about his bank login. The third line is the pitfall above: an ordinary search that fails and reports nothing. The Python docs warn against building a query with string operations for this reason and give the fix, a placeholder in the SQL with the values passed separately to `execute()` (sqlite3 - DB-API 2.0 interface for SQLite databases, The Python Standard Library documentation). The database then treats the values as data, whatever characters they contain. The fix is small, and that makes it tempting to accept the agent's first version of it. Ask for a test that fails on the old code and passes on the new. The review then ends with proof that the fix covers what you found. Scaling the check to how little you watched the agent work is the habit the Academy's *Claude Code in action* course teaches for unsupervised runs (Claude Code in action, Claude Academy), and a failing test is the cheapest proof. #### Checkpoint: Brief the fix Which brief do you give the agent? - Ask the agent to escape every single quote in term before it goes into the query, and to keep the rest. - Ask the agent to add a # nosec comment on the query line, because the route only passes logged-in users. - Ask the agent to make the search secure, and to fix anything else it notices in the files it reads. - Ask for placeholders for both values, errors raised to the caller, and a test with the hostile term and O'Brien that fails on the old code. Here is a brief in that form. The pair below is illustrative: the author wrote both the brief and a summary of the kind of answer to expect. #### Prompt (illustrative, not a recorded transcript) ```text In `notes_search.py`, `search_notes` builds its SQL with an f-string, so a search term such as `%' OR owner != '` returns other users' notes, and a term with an apostrophe fails. Change the query to use `?` placeholders for `owner` and `term`, and pass both values as the second argument to `execute()`. Remove the `except sqlite3.Error` block so errors reach the caller. Move `ADMIN_PASSWORD` to an environment variable. First write `test_notes_search.py` with two tests, the hostile term (expect an empty list for Alice) and `O'Brien` (expect her note), and show me that both fail on the current code. Don't change any other file, and don't add a dependency. ``` #### Response ```text I added test_notes_search.py with the two tests, and both fail on the current code. Then I changed search_notes to use placeholders and removed the except block, and is_admin now reads NOTES_ADMIN_PASSWORD from the environment. Both tests pass. ``` The fixture has the result as `fixed/notes_search.py`, with the test next to the agent's version. `run_test.py` runs the test against each version in its own temporary directory. #### Example: Run the test against both versions From `~/security-review`, run the script and compare what you see with the output below. ```sh python3 run_test.py ``` Output: ```text agent version: FAILED (failures=2) fixed version: OK ``` In a second recorded run the same day, Bandit reported no issue for `fixed/notes_search.py` and exited with status 0. A clean run means that no rule matched. You know the fix works because of the test, and the test is also what stops a later change from bringing the problem back. ## The review and the blast radius The search runs with the app's database connection. Its blast radius is everything that connection can read or change. For a read-only account that's every note in the database. For an account that can write, a string-built query can also delete or change notes. How long you spend on a security review follows the same rule as any risk decision: look at what the code can reach before you decide how much checking it needs. SQL injection and prompt injection are the same mistake in two places. In both, data that should only be read is placed next to instructions, and the reader can't tell them apart. The agent's f-string puts Alice's search text in the SQL. A web page an agent reads puts its text in the prompt. The fix for SQL is to keep data and instructions in separate channels with a placeholder. A prompt has no such channel, which is why an agent's access has to be limited instead. ## Exercise After this exercise you can review an agent's change to code that reads user input without trusting a clean analyzer run. From the root of your clone of the course repository, make two fresh copies of the fixture, one for the agent and one that keeps the agent's original version. Remove the fix and the test from both, so that the agent starts from the change as it arrived. If `~/search-fix` or `~/search-before` exists from an earlier try, delete it first, or `cp` puts the new copy inside the old one. ```sh cp -R site/examples/coding-with-agents/security-review ~/search-fix cp -R site/examples/coding-with-agents/security-review ~/search-before rm -r ~/search-fix/fixed ~/search-fix/test_notes_search.py ~/search-fix/run_test.py rm -r ~/search-before/fixed ~/search-before/test_notes_search.py ~/search-before/run_test.py ``` Next, install Bandit 1.8.6 into a virtual environment in your home directory. A virtual environment is a directory with its own Python packages, and installing into it leaves the course repository and both copies unchanged. First check which Python you have: ```sh python3 --version ``` Bandit 1.8.6 needs Python 3.9 or newer, and its PyPI page lists Python 3.9 to 3.13 as supported (bandit 1.8.6, Python Package Index). On the same day as the recorded run, on Python 3.14.7, it printed two `ERROR` lines for `notes_search.py`, listed none of the findings and exited with status 0, which looks like a clean run. If your version is 3.14 or newer, check the system Python (`/usr/bin/python3` on macOS) with `/usr/bin/python3 --version`. If that prints 3.9 to 3.13, write `/usr/bin/python3` in place of `python3` in the first command below. If it doesn't, skip the install and the Bandit steps, and work from the recorded run in this lesson. If your version is 3.9 to 3.13, or you use the system Python, create the environment, install the pinned version and run it on the agent's version: ```sh python3 -m venv ~/bandit-1.8.6 ~/bandit-1.8.6/bin/pip install bandit==1.8.6 cd ~/search-fix ~/bandit-1.8.6/bin/bandit -q notes_search.py ``` Compare the findings with the recorded run and write them down. Then read the function yourself, list every place where input enters and every `except`, and note which problems Bandit missed. Start a coding agent session in `~/search-fix`, which is a copy you can delete, and give it a brief for the parameterized version with a test, in your own words. Before you accept the change, the agent's test must fail when you copy it into `~/search-before` and run it there. In `~/search-fix` it must pass. Then run Bandit on the agent's new version: ```sh ~/bandit-1.8.6/bin/bandit -q ~/search-fix/notes_search.py ``` B608 is gone when the query uses placeholders. B105 still shows if your brief left the password in the source. A clean run means that no rule matched, and the test is what shows that the fix works. When you are done, including the stretch goal, delete the environment with `rm -r ~/bandit-1.8.6`. A good result is a short note with the Bandit findings, the problem it missed, your brief, and the two test results. The agent's diff changes one function and adds one test file, and it doesn't add a dependency. Reflect: which of the problems would you have found without Bandit, and which one would you have missed without asking what happens when the input is hostile? Stretch: If you installed Bandit above, run it on a public Python project before you delete the environment. Clone one into a scratch directory outside the course repository and run ~/bandit-1.8.6/bin/bandit -r on its source directory, without -q. Read the Files skipped list at the end first: Bandit lists a file there when it can't parse it, for example a file that uses syntax newer than your Python. Pick one finding with Low confidence, and decide in writing whether it is a real problem in that code, and why. ## Recap 1. Agents learn from public code, and they learn the security bugs in it too (Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions, 2022 IEEE Symposium on Security and Privacy (SP 2022), 754-768). Look for unvalidated input, string-built queries and commands, secrets in code, permissive defaults, disabled checks, and error handling that hides failures. 2. A static analyzer such as Bandit finds the patterns it has rules for. Read its findings, including the ones with Low confidence, and then read the code for what it can't know. 3. At each place where data enters, ask what happens when the input is hostile, and follow every `except` to what the user sees. 4. Brief the fix with placeholders and a test that fails on the old code, and run that test on both versions before you accept it (sqlite3 - DB-API 2.0 interface for SQLite databases, The Python Standard Library documentation). 5. The blast radius of an injection is everything the code's connection can reach, and SQL injection is the same mistake as prompt injection: data placed where instructions go. #### Habit The next time you review an agent's change that reads user input, find each place the input enters and ask what happens if it is hostile before you approve. ## References - Security and adversarial thinking, Brilliant, Coding with AI skills map - Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions, 2022 IEEE Symposium on Security and Privacy (SP 2022), 754-768, https://doi.org/10.1109/SP46214.2022.9833571 - Bandit documentation, PyCQA, https://bandit.readthedocs.io/en/latest/ - sqlite3 - DB-API 2.0 interface for SQLite databases, The Python Standard Library documentation, https://docs.python.org/3/library/sqlite3.html - Claude Code in action, Claude Academy, https://academy.claude.com/courses/claude-code-in-action - bandit 1.8.6, Python Package Index, https://pypi.org/project/bandit/1.8.6/