Skip to main content
A custom library lets you assess candidates on the situations they will actually meet in the role. This guide is about writing questions that do that well. For how libraries work, creating one and publishing, see Creating your own question library. A question is good when every candidate, whatever their background, can produce a full answer, and that answer shows their language rather than their knowledge. Almost every tip below follows from that.

Five rules for every question

1

Test the language, not the job

Set the question in your world, but never make the answer depend on knowing it. A new hire should not be marked down for not knowing your procedures yet, or marked up for already knowing them. If a candidate who has never worked in the industry could not answer, rewrite it.
2

Ask for something to say

The score comes from what the candidate produces. “Do you like working with customers?” can be answered in one word, and one word cannot be scored. Ask for a reason, an example, a description or a decision.
3

One task per question

Three questions stacked into one prompt produce three thin answers, and the candidate spends their time remembering the list instead of answering it. Ask one thing and let them go deep.
4

Keep it short to read

Every second spent reading the question is a second not spent answering it. Set the scene in one or two sentences. Use simpler words in the question than you expect in the answer, so a weaker candidate still understands what is being asked.
5

Keep it fair to everyone

Not every candidate has a car, a degree, children, a garden or a holiday abroad. Avoid questions that assume a particular life, culture, religion or politics, and avoid anything a candidate might find personal or upsetting to talk about while being assessed.

One question for every level

Speaking and writing questions are not written for a particular CEFR level. A good open question works for every candidate: someone at A2 understands it and gives a short, simple answer, and someone at C1 answers the same question in far more detail and range. The level comes from how they answer, not from which question they got. A closed question cannot do this. “How many years have you worked?” gets the same few words from a beginner and from a fluent speaker, so it shows nothing about either. “Describe the best job you have had and what you learned from it” can be answered at every level, and the answers are very different. So write questions that a beginner can understand, and leave room for a strong candidate to show more. You do not need a separate library for each level you hire at. Decide the level a role needs, and compare each candidate’s result to it.
Reading and listening work differently: they adapt, starting with easier questions and moving to harder ones depending on the answers. They always use Evalingo’s own questions, so there is nothing to write.

Question by question

Each type measures something different, so each has its own brief. The timings are fixed and shown in the editor beside the question.

Conversation starter

Speaking, section 1. Two are drawn per assessment. It measures whether the candidate can get going and speak fluently on familiar ground.
  • Ask about everyday experience, never knowledge.
  • Make it easy to start and possible to keep going for the full time.

Topic discussion

Speaking, section 2. It measures extended speech: whether the candidate can hold a position, organise it and support it.
  • Ask for an opinion and the reason for it.
  • Choose a topic anyone can have a view on without specialist knowledge.

Scenario response

Speaking, section 2. It measures practical communication: handling a real situation in the right tone, under mild pressure.
  • Say who they are talking to. That decides the register, and register is part of the score.
  • Give them a goal, not just a situation to describe.

Reading fluency passage

Speaking, section 3. It measures pronunciation, stress and intonation, separate from having to think of what to say.
  • Aim for 60 to 90 words; the editor counts as you type. A passage that cannot be finished comfortably turns into a test of reading speed.
  • Use ordinary vocabulary and a mix of punctuation: a question, a list, a short and a long sentence.
  • Avoid names, abbreviations and numbers that could be read more than one way.
Thank you for calling our customer service team. Before we begin, could you please confirm your full name and the email address on your account? I can see that your order was sent on Monday, but it has not arrived yet. I am sorry about the delay. There are two options: we can send a replacement today, or we can refund the full amount. Which would you prefer? Either way, you will receive a confirmation email within the hour.

Email

Writing, section 1. It measures whether the candidate covers what was asked, in the right tone for the reader.
  • Say who the email is to and why.
  • List the points it must cover. That is what lets task response be scored fairly rather than guessed.

Short written response

Writing, section 1. It measures clear, accurate everyday writing.
  • Ask for something with a natural shape: a message, a note, a reply.
  • One task only. There is not time for more.

Extended writing

Writing, section 2. It measures organisation and range: structure across paragraphs, argument and more complex grammar.
  • Ask for a position, supported with reasons. That gives the answer a structure to score.
  • Make it achievable in the time. A three-part task produces three short paragraphs.

Examples by industry

The Evalingo standard library is written for general workplace contexts. A library of your own can put candidates in situations closer to the role, which gives them something familiar to talk about. The same rules apply in every industry: set the scene in your world, but keep the question answerable by someone who has never done the job.

Instructions

The Instructions field appears above the question. It starts with a sensible default in your library’s language, which works. A better one restates the task for that specific question in one short sentence:
  • Default: Answer in a few sentences. Give an example or a reason rather than a one-word answer.
  • Better: Describe your day from start to finish and say which part you enjoy most.

Writing in another language

Write each library in the language it assesses, rather than translating an English question word for word. A translated question often reads as a translation, and can end up easier or harder than the original. Ask a fluent speaker to read every question before you publish it.

How many to write

The minimum is enough to fill every section once, which is six questions. At the minimum, every candidate sees the same test, so aim for at least three times the minimum in each section: six conversation starters, three extended responses, three reading passages, three casual writing tasks and three extended writing tasks. More is better, especially if candidates may talk to each other.

Starting from an AI draft

If you are not sure where to start, Draft with AI in the question editor writes a first version from a topic you describe, following the rules on this page. Treat it as a starting point: check it against the list below like anything you wrote yourself. See Drafting with AI.

Before you publish

Could someone who has never done this job answer it fully?
Does it ask for a reason, an example or a decision, not a yes or no?
Is it one task, and can the question be read in a few seconds?
Would it be fair and comfortable to answer for a candidate from any background?
For scenarios and emails: is it clear who the candidate is speaking or writing to?
Has a fluent speaker of the library’s language read it?
Then try it yourself: send yourself an assessment using the library, and sit it as a candidate would. It is the quickest way to find a question that is too long to read or too thin to answer. Questions about writing your library? Email support@evalingo.ai and we will look at a draft with you.