When in voice search using customized languages (not UI supported languages because it influences the language options in My Account) it is not possible to set a default language: https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/020Primo_VE/Primo_VE_(English)/050Display_Configuration/010Configuring_Discovery_Views_for_Primo_VE#Defining_a_View
I suggest it should be possible to set the default language of Voice Search (as once was possible in earlier versions of VE)