User manuals for software typically make use of terms that were used during screen design for the software. Such lists of terms are especially useful if the software is to be internationalized. For a project I am working on to create such documentation, I was unable to come up with such a list.
Casting about to find a way to address the issue of how to proceed without a list of terms. I contemplated simply compiling a list from the documentation itself, which I did in French. The results were less than satisfactory. My thinking then turned to having the translators for each of the three languages review the first section in hopes that this process might suggest the terms to be added. That effort also met with challenges.
Remembering the 106 screenshots that we have for the software in all four languages, a solution for a reasonable approach began to take shape. Returning to the idea of collecting the needed terms from those screenshots, I quickly realized it would be a daunting task to do this manually. This approach would also be one fraught with the possibility of many mistakes.
I recently posted about my project to do automatic handwriting recognition on a journal my wife kept for many years while our children were young. After setting that aside due to challenges encountered two years ago, I decided to try again a couple of weeks ago and was nothing short of astounded at what I found.
With that in mind last week and wanting to find a solution for the list of terms I needed, it hit me that I could get generative AI tools to do optical character recognition on the 106 screenshots in four languages to produce the needed list. After a quick couple of successful experiments submitting screenshots via the Web interface and retrieving the needed text, I was impressed. That success led to the realization of what it would take to go through that process with the 424 images involved. What was needed was a program to submit screenshots, retrieve the results, and then match terms from English to the other three languages.
I then worked with Google’s Gemini, to set up access to the system’s API (Application Programming Interface). Gemini helped me create a program in Python to submit the 424 images to Gemini’s large language model and receive a total of over 2,000 terms in each language. Then came the daunting task of matching up those terms across languages. I had a couple of unsuccessful attempts, the result of asking the AI to match terms in two lists of over 2000 items each. I suggested a divide and conquer approach, but that did not help. The firstg couple of attempts met with total failure. Indeed, one of the systems AI simply provided some code and a dozen terms. Another simply responded that it could do what was need needed, but it then abandoned the task with no further explanation. Yet another (the Chinese DeepSeek system running on Groq hardware to accelerate processing) generated an incredibly lengthy explanation of the difficulty of what I was asking. The response to the same prompt I had supplied the other systems began with this:
Okay, so I need to figure out how to solve this problem where I have to calculate the number of ways to arrange some objects with certain constraints. Let me try to break it down step by step. First, I remember that when dealing with permutations and combinations, the key is to understand whether the order matters and if there are any restrictions. In this case, the problem involves arranging objects where some are identical and others are different, and there are specific constraints on their positions. Hmm, so let’s say I have a total of n objects. Out of these, there are some identical objects and some that are distinct. For example, maybe I have 3 identical red balls and 2 distinct blue balls. The problem might ask how many ways I can arrange these balls in a line or a circle, considering that some positions are restricted. Wait, the user didn’t specify the exact problem, but I think I should consider a general approach. Let me think of a common scenario. Suppose we have n objects where there are duplicates and some specific constraints, like certain objects must be placed in particular positions or can’t be placed next to each other.
That was the first of FOURTEEN pages of explanation! Needless to say, more work was needed on the approach I was taking. 😊
The good news is that the failures I encountered prompted me to come up with an approach that worked. I returned to Gemini for help creating code in Python to submit the English screenshots one at a time, doing the matching across the other three languages. The procesing included and saving the results into an Excel file, which resulted in 2,382 entries. After using Excel to remove the duplicates, 368 items remained, with some cleanup still needed. Et voilà!!
The following illustrates screenshot illustrates user error on my part. I made a mistake and used an up-to-date set of English screenshots, but the list of screenshots for the other languages were out of date.
This once again proves the old GIGO adage, “Garbage in, garbage out!”
