There is a case-sensitivity issue in my code but I don't know where

[Julia Simon]

Hello. These are the local macros I need for my do-file to work:

* II. Other macros definition
**# If interested in less waves - EDIT HERE
local wave 1986 1992 1998 2004 2010 2014 2018

**# If interested in less tables - EDIT HERE
local tables indiv hh transm

// Individual-related table names
local indiv actfin86 act921 individu INDIVIDU
    
// Household-related table names
local hh actfin86 act921 menage MENAGE
    
// Transmission-related table names
local transm actfin86 act922 transfer transm transmission TRANSMISSION

The last 3 locals refer to filenames that are stored at different places of my working directory. Now have a look at this code:

// Wave-specific macros
foreach wa of local wave {
    
    // The 2010 wave has a different directory than the rest.
    if `wa' != 2010 {
        local data_`wa' "`path'/`wa'/Stata"
    }
    else local data_`wa' "`path'/`wa'/STATA/Methodologie10"
    
    foreach tab of local tables {
        foreach name of local `tab' {
            
            local wrong_files "`data_`wa''/`name'.dta"
            capture: confirm file `wrong_files'
            if !_rc {
                local `tab'`wa': dir "`data_`wa''" files "`name'.dta", respectcase
                local `tab'`wa': list clean `tab'`wa'
                di "`wrong_files'"
                di "``tab'`wa''"
            }
            
            else {
                continue
            }
        }
    }
}

For some reason, when the loop is running for the last elements of my three locals (INDIVIDU, MENAGE, TRANSMISSION), it seems like the code creates empty local. This is the result of the commands I highlighted :

D:/XXXXX/inputs/working_data/2010/STATA/Methodologie10/menage.dta
menage.dta
D:/XXXXX/inputs/working_data/2010/STATA/Methodologie10/MENAGE.dta

D:/XXXXXX/inputs/working_data/2010/STATA/Methodologie10/transmission.dta
transmission.dta
D:/XXXXXX/inputs/working_data/2010/STATA/Methodologie10/TRANSMISSION.dta

D:/XXXXXX/inputs/working_data/2014/Stata/individu.dta

D:/XXXXXXX/inputs/working_data/2014/Stata/INDIVIDU.dta
INDIVIDU.dta

In the 2010 folder, the correct file is menage.dta and transmission.dta. Yet I don't know why the code is creating a blank void local macro for what is supposed to be MENAGE.dta and TRANSMISSION.dta since the files do not exist. Normally the code should be at the "continue" part. In the last two remaining lines, INDIVIDU.dta is the correct file yet it created a blank local for a non-existent file called individu.dta. So I'm thinking there's definitely a case-sensitivity issue in my code but I don't know where. Can someone help me? Thanks!

When I remove the respectcase option the code runs smoothly, but I don't want to do this because there might be Mac users among my working team.

[Mike Lacy]

Can you create and describe a simple example in which this problem occurs, preferably one with a simpler file/folder structure? I suspect that would help both you and others to diagnose the problem.

I also have an intuition that there's a different and easier way to accomplish your overall goal (which I don't understand), so some kind of description of that would be helpful if you'd like to get suggestions in that direction.

[Julia Simon]

Dear Mike,

Thank you for your message. I share the same intuition than you, however considering my coding skills I tried to do my best.

Here's a simple example. Suppose I have three folders, one for 1986, one for 1992, one for 1998. In each of these folders, I have one dataset for households, and one dataset for individuals, but these files can have different and random names. That would make something like:

1986:
- household.dta
- indiv.dta

1992:
- HOUSEHOLD.dta
- INDIVIDU.dta

1998:
- hh.dta
- individu.dta

These discrepancies in names are difficult to deal with when one wants to append files together, so my idea was to create standardized locals for each type of dataset (household/individual), and foreach year, and each of these locals would contain the actual filename. So for instance, if I want to refer to individual datasets, I would just need to use their correspondign local to avoid dealing with hazardous names, e.g. :

the local indiv1986 would contain the filename "indiv.dta"
the local indiv1992 would contain the filename "INDIVIDU.dta"
the local indiv1998 would contain the filename "individu.dta",

and so on. In short, the idea was to replace chaotic file names by standardized local macros, as I received the instruction to keep the original files untouched.

However, the problem is when two files in different folders have the same name, but with different case standards. the individual file in 1998 is written in lowercase, and the individual file in 1992 is written in uppercase. My code asked Stata (1) to create every possible combination of every possible path, even they don't exist (2) to confirm if each of these paths effectively exist using the confirm file command (3) if they exist, to store the name of this path in a local that would later be used for appending purposes.

It seems that the confirm file command is case-insensitive and does not care whether the file is written in uppercase or lowercase. I'm just assuming this, but I have no way of checking that. So my code then creates a local supposed to give the path of a file that does not exist, which is why I get blank locals and my code does not work.

I hope that was clearer, sorry for possible English language mistakes.