aqora test for this challenge, and the score exploded to an extremely large number.events-validation.hdf5 file. Correct me if I'm wrong, the d_sigma in the file is the one used for scoring when running aqora test, and the scoring method is using MAPE. I calculated the MAPE score from my generated predictions with the events-validation.hdf5 data, and it is the same as the aqora test score, i.e. an extremely large value.d_sigma in the events-validation.hdf5 has many rows with very small values and many negative values. For instance, there are 11067 in the file with d_sigma ≤ 0. These negative values are exploding the MAPE.d_sigma? Can it be negative or close to 0?d_sigma ≤ 0 in the training dataset since from what I understand it is a probability and cannot be negative?Posted by eno •
d_sigma, yes, that can happen, even though we would usually regard this value as a probability for the event to happen. But it's a bit more complicated than this in practice. The negative values here come from the so-called parton density function, which can become negative (at higher orders in perturbation theory). Note that this is only a statement about individual points in the sample. Overall, with enough statistics, you will always get a positive sigma for any meaningful physics observable.Posted by grossim •
Posted by jag •
d_sigma in my predictions and the validation file and then computing the MAPE score based on that?Posted by grossim •
Posted by jag • (edited)
import numpy as np
from sklearn.metrics import mean_squared_error, mean_absolute_error, mean_absolute_percentage_error, r2_score
mse = mean_squared_error(y_targets, y_preds)
rmse = np.sqrt(mse)
mae = mean_absolute_error(y_targets, y_preds)
mape = mean_absolute_percentage_error(np.abs(y_targets), np.abs(y_preds))
r2 = r2_score(y_targets, y_preds)
print(f"RMSE: {rmse:.4f}")
print(f"MAE: {mae:.4f}")
print(f"MAPE: {mape:.2%}")
print(f"R²: {r2:.4f}")