Overshoulder

Your agent works in its window. Your desk stays yours.

Overshoulder is an open-source MCP server for Windows. An AI agent takes one app window, sees its controls, and clicks and types inside it in the background while you keep your own mouse and keyboard. It can record what it did as a tutorial with numbered steps.

git clone https://github.com/drknowhow/overshoulder

Find app profilesSource on GitHub

A tutorial Overshoulder recorded while an agent worked out a 15% tip on 48.50 in Calculator.
The steps
  1. Click Clear
  2. Click Four
  3. Click Eight
  4. Click Decimal separator
  5. Click Five
  6. Click Zero
  7. Click Multiply by
  8. Click One
  9. Click Five
  10. Click Percent
  11. Click Equals

Profiles on the registry

A profile is what agents learned about one Windows app: its windows and controls, which methods work there, its shortcuts and quirks, and short how-tos. Overshoulder hands it to an agent when the agent takes a window of that app, so it starts with what earlier agents learned. A profile is information: it never lets an agent do more than you allow.

To use one, download it and open the file. Overshoulder shows everything in it and adds nothing until you say yes.

Install

Overshoulder runs on Windows 10 and 11. It needs Python 3.11 or later, and ffmpeg for recording.

git clone https://github.com/drknowhow/overshoulder
cd overshoulder
python -m venv .venv
.venv\Scripts\pip install -e .

Then start it and connect an agent; the quick start covers both. Any agent that speaks MCP can use it.

Share what your agents learned

  1. Export the profile: overshoulder profile export <app>, or Export on the settings page. An export leaves out window titles, typed text, file names and anything seen in only one session.
  2. Open a pull request on the registry that adds it at apps/<app id>/<your GitHub login>.osp.
  3. Checks run on the pull request. Then Yep reviews it and merges it, the registry signs it, and it appears here.

What the checks refuse. Profiles are in the public domain under CC0, so anyone may use them for anything.