Can a small language model turn everyday requests from across Africa into the right actions?

AI assistants are moving beyond answering questions. Instead of telling you how to buy airtime, an assistant can buy the airtime for you. Instead of explaining how to send money, it can send the money.

To do this, an assistant needs to use tools. A tool is a function that lets an AI system take an action or look up information, such as send_money(recipient, amount, currency, provider). The model has to work out which tool to use and what information to give it.

In this hackathon, you will train a small language model that takes an everyday request from across Africa, looks at the tools available to it, and produces the correct tool call.

{
  "tool": "send_money",
  "parameters": {
    "recipient": "Tunde",
    "amount": 5000,
    "currency": "NGN",
    "provider": "MTN MoMo"
  }
}

Why small language models?

Large language models can be very capable at tool use, but they are expensive and difficult to run in resource-constrained environments. This hackathon asks a different question: how capable can a small language model become when we train it specifically for tool calling?

You will work with open models of up to 4 billion parameters and experiment with fine-tuning, synthetic data generation, prompting and inference strategies. The goal is not to build the biggest model. The goal is to build the most accurate small one.

This hackathon is designed to help you gain practical skills for building and studying language models. You keep these whether or not you top the leaderboard.

• Tool calling

  Turn natural language into structured function calls, with the right tool and every parameter.

• Fine-tuning small models

  Adapt open models of up to 4B parameters with LoRA, on a free GPU.

• Data-centric AI

  Generate synthetic data, mine hard examples, and grow the training set where your model fails.

• Evaluation and error analysis

  Find which tools, parameters and kinds of request break your model, and fix them.

No prior competition experience is assumed. The starter notebook walks you through prompting and fine-tuning Gemma 3 270M, and writes a submission you can upload.

The dataset maps English-language requests in African contexts to the right tool and its parameters. Each example has a request, 4 to 6 available tools (the relevant ones and some distractors), and the answer.

How the splits work

Train 15,000 examples

Request, tools and answer. Use it to train and validate your model.

Dev 2,000 examples

Request and tools, with the answers hidden. Submit predictions to see your score on the development leaderboard.

Test 4,000 examples

Request and tools, with the answers hidden. Released when the final phase opens, and used for the final ranking.

The dev and test sets use different names, neighbourhoods, towns, markets and wording from the training data. The aim is to test whether your model has learned tool calling rather than memorised examples.

Three kinds of request

Where the requests come from

The language is English, but the world in the data is African: local expressions and chat shorthand, names and titles, mobile money providers and networks, currencies, markets, dates and times.

• Nigeria
• Ghana
• Kenya
• Uganda
• Tanzania
• Rwanda
• South Africa
• Zambia
• Ethiopia

All names, numbers and codes are made up. The full labelling rules and value formats (amounts as numbers, ISO currency codes, phone numbers as digits, dates such as 15 March, times such as 19:00) are in the dataset README.

Get the data on GitHub Read the labelling rules

Out of the box, small models are poor at this task. Without training, Gemma 3 270M gets almost no requests right, and even Gemma 3 4B gets about a third (34% on a 300-request sample of dev). Fine-tuning changes that.

Even the smallest Gemma model goes from almost nothing to 88% with fine-tuning. How close to 100% can you get?

What might you explore?

• FunctionGemma 270M
• Larger models, up to 4B
• LoRA / QLoRA
• Synthetic data
• Hard-example mining
• Data curation
• Knowledge distillation
• Few-shot prompting
• Self-consistency
• Test-time scaling
• Ensembling

Try these, combine them, or develop your own idea. Try, evaluate, improve, repeat.

There is one metric. A prediction is correct only when the selected tool and all of its parameters match the answer. When a request needs several calls, every call must be correct and in the correct order.

What counts as wrong

If the answer is send_money to Tunde for 5000 NGN on MTN MoMo, then each of these predictions is wrong:

• The right tool, but amount: 500.
• The right tool and amount, but the wrong provider.
• A value that is not in the request, such as a provider the user never mentioned.

How answers are compared

• Text ignores case, punctuation and extra spaces, so "m-pesa" matches "M-Pesa".
• Numbers are compared as numbers: 5000 matches 5000.0, but the text "5000" does not.
• A parameter left out counts as null.
• A list containing one call is the same as that call.

The hackathon runs on Codabench. You upload a ZIP file with your predictions, and your score appears on the leaderboard once it has been scored.

Rules

• Use any open-weight model with at most 4B parameters. Gemma 3 270M, 1B and 4B, and FunctionGemma 270M are recommended.
• No API calls or larger models may be used when generating predictions.
• Any training technique is allowed, including fine-tuning, synthetic data generation, prompting and decoding strategies.
• You may augment the training data.
• You may not manually label development or test examples.

Submission process

1. 1
   Join the competition

   Create a Codabench account and register for the hackathon.

2. 2
   Train your model

   Start from train.jsonl and use any technique within the rules.

3. 3
   Predict every request

   Make one prediction for every id in dev.jsonl (in the final phase, test.jsonl).

4. 4
   Write predictions.json

   Map each id to its tool call, or to a list of calls when the request needs several actions.

5. 5
   Zip it

   Place predictions.json at the top level of a ZIP file.

6. 6
   Upload

   Upload the ZIP under “My Submissions” on the Codabench competition page.

{
  "dev-01234": {
    "tool": "send_money",
    "parameters": {"recipient": "Aisha", "amount": 2500,
                   "currency": "NGN", "provider": "Airtel Money"}
  },
  "dev-01235": [
    {"tool": "check_balance",
     "parameters": {"provider": "M-Pesa"}},
    {"tool": "send_money",
     "parameters": {"recipient": "Otieno", "amount": 500,
                    "currency": "KES", "provider": "M-Pesa"}}
  ]
}

The hackathon has two phases. All times are in UTC.

Development phase 10 Oct – 7 Nov 2026

Train your model, predict dev.jsonl, and climb the development leaderboard. Closes at 00:00 UTC on 8 November.

Final phase 9 – 10 Nov 2026

test.jsonl is released. Submit your predictions for it; the final ranking uses exact-match accuracy on the test set. Opens at 00:00 UTC on 9 November and closes at 00:00 UTC on 11 November.