We release the results CSV files for each of the experiments.
| Type | Data |
|---|---|
| Results | results.tar.xz (md5: 6f6de4da1f6c98b31b188e829c19f293) |
| results.tar.xz.part.000 results.tar.xz.part.001 results.tar.xz.part.002 results.tar.xz.part.003 results.tar.xz.part.004 results.tar.xz.part.005 results.tar.xz.part.006 results.tar.xz.part.007 results.tar.xz.part.008 results.tar.xz.part.009 results.tar.xz.part.010 results.tar.xz.part.011 results.tar.xz.part.012 results.tar.xz.part.013 results.tar.xz.part.014 | |
| Aroma scores | results-aroma.tar.xz (md5: d9dcce739d1a4f030049ea872d9a7c17) |
| results-aroma.tar.xz.part.000 results-aroma.tar.xz.part.001 |
Each of the CSV files comprehends the following data:
| Field | Description |
|---|---|
size | The evaluation strategy, between 1k and full. |
fold | The cross-validation fold number. |
stackoverflow_question_id | The question ID of the StackOverflow question. |
stackoverflow_answer_id | The answer ID of the correct code snippet on StackOverflow. |
predicted_rank | The rank value predicted by the model for the correct code snippet. |
predicted_distance | The computed cosine distance, used to compute the rank for the correct code snippet. |
predicted_first_distance | The cosine distance computed for the instance that the model ranked as first. |
predicted_first_stackoverflow_answer_id | The answer ID on StackOverflow for the instance that the model ranked as first. |
predicted_last_distance | The cosine distance computed for the instance that the model ranked as last. |
predicted_last_stackoverflow_answer_id | The answer ID on StackOverflow for the instance that the model ranked as last. |
distance_mean | The mean of the cosine distance values computed for all the test code snippets. |
distance_std | The standard deviation of the cosine distance. |
distance_min | The minimum value of the cosine distance. |
distance_q1 | The first quartile of the cosine distance. |
distance_q2 | The second quartile (median) of the cosine distance. |
distance_q3 | The third quartile of the cosine distance. |
distance_max | The maximum value of the cosine distance. |
aroma | The aroma score value, only in results-aroma.tar.xz. |