Consultancies using rockfall simulation software, as well as rockfall software developers, have been invited to use their simulation tools to predict the trajectories of 100 rocks in 2D or 3D using a digital elevation model of a site in the French Alps. These data have been compared with data obtained by real size rockfall experiments carried out the same site. Additional data provided to the participants were: the geographic location of the experimental site, the form and volume of the rocks used during the experiments and the locations of two calculation screens on the main path. Characterisation of the soil had to be done by the participants. At the calculation screens, each candidate had to calculate the mean and maximum velocity, kinetic energy and jump height of each rock. In addition, the stopping points of each rock had to be calculated. Eventually, 12 out of 17 candidates from 4 different countries sent back their results. Only 3 out of 12 were capable to simulate the same rockfall kinematics and trajectories with an error of ± 20%. Seven participants were capable to simulate the observed stopping distance with an error of ± 10%. The maximum errors were in the order of 400%. Among the commercial models involved, three of them were used by multiple participants. The outcomes of the test showed that two different users can obtain invalid or very accurate results with the same model. This indicates that the role of the expert is crucial in hazard expertises that use rockfall simulation models. Since this has been the first benchmarking test of its kind, the outcomes should be used with caution, as only one test site has been used. At another test site, results could be completely different. For natural hazard modelling in general, there is a need for accurate validation data.