fb392c8d-10a2-4f63-8731-3b51e0b3515420220114082719040naun:naunmdt@crossref.orgMDT DepositInternational Journal of Education and Information Technologies2074-131610.46300/9109http://www.naun.org/cms.action?id=3037111202211120221610.46300/9109.2022.16https://npublications.com/journals/educationinformation/2022.phpAmineSaboniData User-Centered Tribe OCTO Technology Paris, FranceMohamed RidhaOuamaneDepartment of Electrical Engineering PRISME Laboratory, INSA Centre Val de Loire Bourges, FranceOuafaeBennisDepartement of Automatic PRISME Laboratory, University of Orleans Chartres, FranceFrédéricKratzDepartement of Automatic PRISME Laboratory, , INSA Centre Val de Loire Bourges, FranceThis article investigates a methodology to design an automated supervision report, ensuring the suitability between the designers and the users of an algorithm. For this purpose, we built a super-vision tool, focused on error diagnosis. The argumentation of the article relies first on the exposition of the reasons to use model reports as a supervision artefact, with a prototype of implementation at an organization level, describing the necessary tooling to industrialize its production. Finally, we propose a method for supervising machine learning algorithms in a responsible and sustainable way, starting from the conception of the algorithm, along its development and dur-ing its operating phase.1142022114202250545https://npublications.com/journals/educationinformation/2022/a102008-005(2022).pdf10.46300/9109.2022.16.5https://npublications.com/journals/educationinformation/2022/a102008-005(2022).pdf10.1145/3449081Rakova, B., Yang, J., Cramer, H., & Chowdhury, R. (2021). Where responsible AI meets reality: Practitioner perspectives on enablers for shifting organizational practices. Proceedings of the ACM on HumanComputer Interaction, 5(CSCW1), 1-23. 10.1038/s42256-019-0088-2Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389-399 Responsible AI widgets for Error Analysis, Microsoft Research Shapash responsible AI tooling (2020), MAIF Nushi, B., Kamar, E., & Horvitz, E. (2018, June). Towards accountable ai: Hybrid human-machine analyses for characterizing system failure. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (Vol. 6, No. 1) 10.1145/3287560.3287596Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., ... & Gebru, T. (2019, January). Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency (pp. 220-229) Salesforce, Model Cards for ai model transparency (2020) https://blog.einstein.ai/model-cards-for-ai-model-transparency/ People+AI Research Group, Google, user needs https://pair.withgoogle.com/chapter/user-needs/ 10.1109/tpwrs.2006.873109Davidson, E. M., McArthur, S. D., McDonald, J. R., Cumming, T., & Watt, I. (2006). Applying multi-agent system technology in practice: Automated management and analysis of SCADA and digital fault recorder data. IEEE Transactions on Power Systems, 21(2), 559-567 Victor Schmidt, CodeCarbon: Estimate and Track Carbon Emissions from Machine Learning Computing, https://github.com/mlco2/codecarbon Bourdon, A., Noureddine, A., Rouvoy, R., & Seinturier, L. (2013). Powerapi: A software library to monitor the energy consumed at the process-level. ERCIM News, 2013(92). 10.3389/fcomm.2020.598454Hoang, L. N. (2020). Science communication desperately needs more aligned recommendation algorithms. Frontiers in Communication, 5, 115 Tournesol public wiki homepage, https://wiki.tournesol.app/index.php/Main_Page 10.1145/3411764.3445188Upol Ehsan, Q. Vera Liao, Michael Muller, Mark O. Riedl, and Justin D. Weisz. (2021). Expanding Explainability: Towards Social Transparency in AI systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 82, 1–19. Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and devops: Building and scaling high performing technology organizations. IT Revolution. 10.1787/gov_glance-2013-graph4-frBlangero, Grimprel, La confiance des utilisateurs dans les systèmes impliquant de l’IA, 2019 https://blog.octo.com/la-confiance-desutilisateurs-dans-les-systemes-impliquant-de-lintelligence-artificielle/ 10.1109/wain52551.2021.00026Muccini, H., & Vaidhyanathan, K. (2021). Software Architecture for ML-based Systems: What Exists and What Lies Ahead. arXiv preprint arXiv:2103.07950. 10.1016/b978-2-294-76753-1.00115-6Lachheb, I. (2021) Le feature store, nouvel outil pour les projets de data science, https://blog.octo.com/le-feature-store-nouvel-outil-pourles-projets-data-science/ 10.1145/3411764.3445518Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P. K., & Aroyo, L. M. (2021). " Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AI. Ait Bachir, S. (2021) Les tests automatisés en delivery de machine learning https://blog.octo.com/les-tests-automatises-en-delivery-demachine-learning/ 10.1109/bigdata.2017.8258038Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017, December). The ml test score: A rubric for ml production readiness and technical debt reduction. In 2017 IEEE International Conference on Big Data (Big Data) (pp. 1123-1132). IEEE. Histoire d’une architecture émergente, Compte-rendu du talk d’Emmanuel Lin Toulemonde, Duck Conf 2021, Alessandro Mosca. Atelier de matrice d’erreur, publication in progress at https://blog.octo.com 10.1609/aaai.v33i01.33012429Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019, July). Updates in human-ai teams: Understanding and addressing the performance/compatibility tradeoff. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, No. 01, pp. 2429-2437). Kumar, I. E., Scheidegger, C., Venkatasubramanian, S., & Friedler, S. (2020, January). Shapley Residuals: Quantifying the limits of the Shapley value for explanations. In ICML Workshop on Workshop on Human Interpretability in Machine Learning (WHI). 10.37394/232018.2020.8.9Kara Combs, Mary Fendley, Trevor Bihl, A Preliminary Look at Heuristic Analysis for Assessing Artificial Intelligence Explainability, WSEAS Transaction on Computer Research, Volume 8, 2020, pp. 61- 72. 10.1561/2200000083Peter Kairouz, H. Brendan McMahan, Brendan Avent, et al, Advances and open problems in federated learning, arXiv preprint arXiv:1912.04977, 2 10.1038/s41746-020-00323-1RIEKE, Nicola, HANCOX, Jonny, LI, Wenqi, et al. The future of digital health with federated learning. NPJ digital medicine, 2020, vol. 3, no 1, p. 1-7. ASLETT, Louis JM, ESPERANÇA, Pedro M., et HOLMES, Chris C. A review of homomorphic encryption and software tools for encrypted statistical machine learning. arXiv preprint arXiv:1508.06574, 2015. 10.4000/books.pressesenssib.4488Roussel F., Pemodjo M., Saboni A. L’atelier matrice d’erreur, démystifier les performances du ML avec ses utilisateurs, https://blog.octo.com/latelier-matrice-derreur-demystifier-lesperformances-du-ml-avec-ses-utilisateurs/