LLMs Are Not Good For Basic Homework

Thu, Mar 6, 2025

My 12 years old child recently had a presentation to do with 3 other children for school, the subject was food and health. It was particularly difficult to organize the children and one of the children came with a full presentation with text, generated by an LLM (possibly corrected by the parents). At first, I thought this was just cheating and completly useless. Furthermore, he failed to read properly the details of the subject, and much of it wasn’t relevant.

Then they did a second round together, at another parents home. And the outcome was even worse, in that it was even further away from the subject and had equally strong use of LLM.

I decided to try out some LLM queries on this subject (in French) after I installed a few local LLMs via Ollama. Ollama makes it really easy to run an LLM locally. The drawback is that the models have not necessarily been trained very recently, for example Llama 3.2 was trained in 2021 according to itself. I tried Llama 3.2 and Mistral, with respectively 3B and 7B parameters. This is another drawback, it is challenging to run much larger models as they would require too much RAM. Ollama github page mentions 32 GB to run the 33B models.

The answers we get are not bad, but they are not great either. For example to the question

En quoi les besoins alimentaires dépendent de l’age, du sexe et de l’activité physique de chaque individu?

Les besoins alimentaires de chaque individu peuvent être influencés par différents facteurs tels que son âge, son genre, sa taille, son poids, son niveau d’activité physique et son état de santé.

Par exemple, les enfants ont besoin de plus de calories et de nutriments pour leur croissance rapide. De même, les femmes enceintes et allaitantes ont des besoins alimentaires différents car elles devront fournir les éléments nécessaires à la formation du fœtus ou au lait maternel.

Les hommes et les femmes ont également des besoins alimentaires différents en raison de leurs besoins nutritionnels uniques, tels que la besoin plus élevé de protéines chez les hommes en raison de leur plus grand poids moyen. En outre, certains nutriments sont spécifiquement importants pour des groupes spécifiques d’individus, par exemple, les femmes en âge de procréer ont besoin d’un apport plus élevé en fer et folate pour s’assurer que leur fœtus ait un développement normal.

Enfin, les individus qui sont plus actifs physiquement ou ceux qui ont une activité physique professionnelle exigeante ont des besoins alimentaires différents car ils nécessitent plus d’énergie pour maintenir leur niveau d’activité. Ils doivent consommer davantage de calories et de nutriments pour répondre à leurs besoins énergétiques supérieurs, ce qui peut inclure des aliments riches en protéines, vitamines et minéraux.

En bref, les besoins alimentaires de chaque individu peuvent être influencés par différents facteurs tels que son âge, son genre, sa taille, son poids, son niveau d’activité physique et son état de santé. Il est donc important de comprendre ces facteurs pour assurer que nous consommons une alimentation adaptée à nos besoins nutritifs uniques.

The introduction and the conclusion are the same. Closely repeated words are another issue: spécifiquement. Furthermore, this word is likely a bad translation from English in the sentence. “spécifiquement importants” does not sound quite correct in French. Finally, it’s funny that they always produce the same structure of answer, small intro, a few points, and “En bref…”. At first it looks good, but very quickly, you understand it as noise. In fact if we analyze the points given as answer by LLMs, we start to notice that they are not as distinct as their presentation and wording suggest. It is not uncommon to see very similar sentences in distincts enumerated points.

Google’s internal LLM is not great either as it does not necessarily give the correct context for understanding.

It makes you believe that 1g of protein has 4 calories. But which kind of calorie? the large Calorie (kcal or C) or the small calorie (c) ? The reality is 4000 c or 4 kcal. Btw, I did not know both were introduced by French scientists, and that the large calorie variant was from Marcelin Berthelot. This is a neat feature of websites or books compared to LLMs, you get more contextual information.

The main problem is that the LLMs tend to bring those lists on you, splitting into categories it decided itself, which are most of the time, not categories most human beings would choose. Here it is to the point that children forget about the actual detailed guidelines given to them, because any of those LLM answers look so convincing. This leaves me with the impression that if they had had no LLM, no internet, no computer, they would have produced a better output.