All national or regional operators within a country who wish for their approach to Participatory Research Participatory Research see : http://www.voyages-scientifiques.com to be represented within the International Standard for Participatory Research Participatory Research see : http://www.voyages-scientifiques.com can join the initiative by:

• registering on the list of Science Education operators
• sharing the models, tools, and processes they would like to see included in the International Standard for Participatory Research
• participating, if they wish, in national consultation meetings held annually as part of the National Science Voyage and Participatory Research Fair in their country
• participating, if they wish, in international consultation meetings held every December at the United Nations in Geneva.

 How can you tell whether scientific research is new and constitutes genuine research?

 The Research Monitoring Tool for participatory research, explained simply

When a project claims that it is conducting scientific research, it is not always easy to know what that means in practice.

There may be a partner university, a recognized researcher, a protocol, hundreds of participants, large amounts of data, conferences and publications. All these elements can be useful, but none of them is sufficient to demonstrate that research is genuinely taking place.

The question this tool seeks to address is much simpler: what is actually happening, from a research perspective, within this project?

It is about looking at facts rather than outward signs of scientific legitimacy. A small association can conduct genuine research. Conversely, a project led by several prestigious institutions can accumulate data for years without managing to analyze them or produce new knowledge.

The starting principle is therefore simple:

evaluate what is actually being done, rather than the promises (status, prestige...) of those doing it.

 Recognizing research through how it operates

Research generally begins with something that is not yet sufficiently well understood.

It may involve understanding why a phenomenon occurs, determining the properties of a material, testing a hypothesis, identifying a species, improving a technical device, studying a population, solving a mathematical problem or understanding data that cannot yet be explained.

The subject of the research can vary enormously depending on the discipline. Yet its overall functioning has common features.

You need to know what you are looking for.

You need to examine what is already known, so as not to spend several years rediscovering something that is already known.

You need to determine how you are going to try to obtain an answer.

You then need to produce something that makes progress possible: observations, measurements, experiments, calculations, simulations, samples, corpora, prototypes, reasoning or proofs.

Then you need to examine what has been produced and determine what it actually allows you to conclude.

It is this concrete chain that the tool seeks to track.

 Existing reference frameworks already exist, but they address different questions

International reference frameworks for defining or evaluating research have existed for a long time.

The Frascati Manual, used internationally to define research and development, focuses in particular on novelty, creativity, uncertainty, the systematic nature of the approach, and the ability to transfer or reproduce the knowledge produced.

Research funding agencies, such as the ANR in France, the SNSF in Switzerland, the NSF in the United States or the European Research Council, also examine the quality of the scientific question, originality, methodology, feasibility and team capacity.

Other approaches, such as DORA or CoARA, seek to improve the way research is evaluated. In particular, they remind us that it is dangerous to automatically use the number of publications, the prestige of journals or certain bibliometric indicators as substitutes for the actual quality of scientific work.

Citizen science and Participatory Science also have their own reference frameworks, particularly around ECSA and Open Science.

The tool presented here does not replace any of these systems. It seeks to solve a more practical problem: to enable someone designing, conducting or monitoring a project to simply verify that research genuinely exists and is continuing to function effectively.

 Observing a few facts rather than completing a very long questionnaire

A highly detailed assessment may include dozens of criteria. This is sometimes necessary, but it quickly becomes difficult to use on a daily basis.

The idea is therefore to look for simple facts that provide information about several dimensions at once.

Suppose, for example, that a project claims to seriously monitor the quality of its measurements.

Rather than starting with a long series of administrative questions, it can be very informative to look at the most recent measurement considered questionable, understand why it was rejected or retained, and see what decision was made on the basis of that difficulty.

This small check already provides a great deal of information.

It shows whether the data are actually being examined, whether the team knows how to detect an anomaly, whether it has quality-control criteria, whether it reflects on its results, and whether it knows how to adapt its method when a problem arises.

The tool relies heavily on this type of observation.

 Cross-checking rather than accumulating

A single piece of information is not always enough.

A project leader may explain that participants contribute to data analysis. This statement may be accurate, imprecise or simply reflect what was originally planned in the project.

You can then look at a session report, examine an output resulting from the work, and ask a participant what was actually analyzed.

When several independent elements tell the same story, the diagnosis becomes much stronger.

When the accounts diverge, that divergence itself becomes interesting. It indicates that one aspect of the project’s operation deserves closer examination.

This logic can be summarized as follows:

Measure less. Observe better. Cross-check more.

The objective is therefore not to obtain one hundred indicators every time ten well-chosen observations can provide the same level of understanding.

 The main questions that almost always remain the same

Whatever the discipline, the tool essentially seeks to track a few fundamental functions of research.

What are we trying to find out?

There must be a question, a problem, an unknown or something to establish.

What do we already know?

The research must be able to position itself in relation to existing knowledge.

How are we looking for the answer?

There must be a method appropriate to the question.

What is actually being produced?

This can take extremely different forms: data, measurements, observations, experiments, simulations, calculations, prototypes, models, corpora or proofs.

How do we verify the quality of what is produced?

The answers will vary depending on the discipline: data quality control, calibration of an instrument, repetition of an experiment, analysis of a bias, verification of a calculation, comparison with a reference, validation of a model.

What do the results actually allow us to understand?

A large quantity of data does not yet constitute knowledge. The data must be analyzed, interpreted and linked to the conclusions they support.

What does what we have just learned change?

This final question is particularly important for tracking research over time.

 Genuine research does not necessarily follow its plan without ever deviating from it

A scientific project begins with a method and objectives. Yet reality does not always behave as expected.

An instrument may work less well than expected. A hypothesis may be refuted. An unexpected phenomenon may appear. A method may prove inadequate. Initial observations may lead to a better question than the one asked at the outset.

Properly conducted research must be able to identify these situations and draw consequences from them.

Changing a protocol is therefore not necessarily a sign that the project was poorly designed. On the contrary, it may show that the team is genuinely observing what is happening and learning from its results.

What matters is understanding why the change was made and what facts it is based on.

The tool therefore seeks to track not only what was planned, but also how the research reacts to what it discovers.

 Negative results are also part of research

Research does not need to confirm its initial idea in order to be successful.

Discovering that a technique does not work, that a material does not have the property being sought, or that a hypothesis is incorrect can constitute important knowledge.

What matters is being able to explain what was tried, what was observed and why the results support that conclusion.

The quality of research should therefore not be confused with its ability to systematically produce the result that was hoped for.

 Citizen science, Participatory Science and participatory research

When people who are not professional scientists take part in research, another question becomes necessary: which scientific functions do they actually perform?

In a citizen science program, participants may, for example, carry out observations or produce data according to a protocol designed by professional researchers. Their contribution may be essential and scientifically very important, even if the analysis and decisions remain mainly in the hands of researchers.

In Participatory Science, participants may be involved in more stages: contributing to analyses, discussing certain anomalies, participating in interpretation or proposing methodological improvements.

In more comprehensive participatory research, professionals and non-professionals effectively contribute to advancing the scientific cycle itself. They may participate in the evolution of questions, the method, the interpretation of results and decisions concerning the next stages of the research.

The tool therefore does not need to rely on the category claimed by the project. It can concretely examine who performs which scientific functions.

This distinction does not establish an automatic hierarchy. Well-designed citizen science can be perfectly suited to its objective and produce excellent results without being intended to become participatory research.

 A tool that must work across all disciplines

Citizen science and Participatory Science are often associated with biodiversity observations, astronomy or environmental monitoring. These are fields in which they have developed extensively, but there is no reason to limit the logic of participation to these disciplines.

Research can take place in a forest, but also in a chemistry laboratory, on a computer, in front of a laser bench, in a prototyping workshop, with genetic data, on materials or around a mathematical problem.

The quality-control techniques will be different.

In water research, attention may be given to sampling quality and sensor calibration.

In chemistry, it may be necessary to examine contamination, standards and repeatability.

In experimental physics, measurement uncertainties and metrology may be decisive.

In computer science, this will include data, code, versions, benchmarks and tests.

In mathematics, attention will focus on assumptions, reasoning, counterexamples and verification of the proof.

The technical vocabulary changes, but the overall structure remains comparable: a question, an approach, elements that make it possible to answer it, a way of verifying their quality, and reasoning that makes it possible to produce knowledge.

 Participating in research does not mean operating every instrument

This distinction becomes important in highly technical disciplines.

A person participating in materials research does not need to operate a synchrotron themselves. A participant in genetic research does not necessarily have to carry out all the sequencing operations themselves. A child participating in optics research obviously should not handle dangerous equipment.

Professional scientists themselves routinely use infrastructures, instruments or analyses operated or carried out by other specialists.

Scientific participation should therefore be assessed through the functions actually performed: understanding what is being produced, contributing to questions or methods when relevant, interpreting results, participating in scientific decisions.

Physical operation of every instrument is not, in itself, a measure of participation in research.

 The same tool before, during and after

One of the main ambitions is not to create one form for designing the project, a second for monitoring it and a third for producing its report.

The main questions remain the same. Their wording simply changes according to the stage of the project.

Before launch, the tool looks at the reality of the preparation.

Does the scientific question exist? Has the knowledge already available been examined? Is the method sufficiently defined? Have certain difficulties already been tested? Are the skills actually required available?

The aim is to evaluate what already exists, rather than to value promises.

During the research, intentions gradually become secondary. The tool looks at what is actually happening: data obtained, problems encountered, analyses carried out, decisions made, changes in method and learning.

After a phase, it becomes possible to examine what was actually produced, what the results genuinely allow us to conclude, what remains uncertain and what the next cycle should investigate.

The project is therefore monitored as a trajectory, rather than as a simple snapshot.

 Evaluating in order to support

The tool is not intended simply to assign a color or a score.

A project may have a very interesting scientific question but a method that is still insufficiently developed. Another may produce good data but may not have organized their analysis. A third may be scientifically robust but need a particular skill to resolve a difficulty.

In these situations, the purpose of evaluation is first and foremost to identify precisely what needs to be improved.

It may lead to continuing the project as planned, strengthening a particular aspect, looking for a mentor, changing a method, reorganizing the research, launching a new cycle or sometimes deciding to stop it.

Stopping is not necessarily a finding of failure. A question may have been answered, a method may have reached its limits, or another organization may now be better placed to continue the work.

 Capitalizing on what works

Research produces knowledge about its subject, but it can also produce knowledge about how to conduct research.

A particularly effective way of checking data with participants, a simple way of organizing analyses, a method for tracking changes to a protocol, or a technique for quickly identifying an anomaly may be useful to other projects.

The tool should therefore make it possible to identify these practices, understand the conditions under which they work and preserve them.

Monitoring then also becomes a way of modeling and transmitting methods that work, rather than starting every new project from scratch.

 What this tool does not seek to evaluate

Research raises many other essential questions.

It must be examined from an ethical perspective.

Epistemological positioning, relationships between different forms of knowledge, the way participants are involved, their learning, the human dimensions of participation, and the project’s impacts on the Sustainable Development Goals also require specific assessments.

These dimensions are addressed by related tools.

This one remains focused on a specific question:

does scientific research genuinely exist within the project, how does it operate, what does it produce and how does it evolve?

 A simple tool does not mean simplified research

You do not need to be a specialist in every discipline to ask certain fundamental questions.

It is possible to ask what the research is trying to understand, what has actually been produced, how a problem was detected, what the data changed or what evidence supports a conclusion.

When the question becomes highly specialized, disciplinary expertise naturally becomes necessary.

The objective is therefore to mobilize the right level of expertise at the right time, rather than turning every evaluation into a heavy audit.

An operator can use the tool for self-assessment. A monitor can use it in the field or remotely. A participant may provide an observation that no one else had noticed. A specialist can be called upon when the difficulty requires their expertise.

Robustness then comes less from the number of boxes completed than from the ability to cross-check facts that are sufficiently representative of how the research actually operates.

 Ultimately, a very simple question

Behind all the methods, disciplines and levels of technicality, the principle remains fairly stable.

When a project claims to have produced knowledge, it must be possible to trace the path leading to that claim.

What was it trying to understand?

What did it know beforehand?

What did it actually do?

What did it observe or produce?

How did it check the quality of what it produced?

What do these elements actually allow it to conclude?

What did it learn?

And what does that learning change in the next stages of the research?

The Research Monitoring Tool simply seeks to make this path visible, early enough to help build the project, easily enough to be used while it is being carried out, and robustly enough to make it possible to evaluate it afterwards and pass on what has been learned.

Ultimately, much of its usefulness lies in an ordinary question that anyone can understand:

How do we know that what we think we have discovered is genuinely what the research allows us to say?