Fix a (potential) issue with log paths on Windows?

[bodkan]

See this post: DReichLab/AdmixTools#63 (comment)

[bodkan]

This is really strange. I asked a colleague with a Windows machine to do a couple of quick checks for me and we don't seem to be able to reproduce the issue. What I thought might be a problem (fixed unix-like path separator in admixr forced on Windows) does not actually seem to be an issue. What's more, I have heard from other people who actually do use Windows with admixr?

Of course, hard to tell without having more information. I will leave this open for now.

[ningyusino]

Hi, @bodkan , glad to hear some more about this issue. Great thanks for your time and effort.
Here is some information I think might help:

• Win10 is my main system. I use Ubuntu 18.04.1 as a subsystem to maintain some biosoftwares which are only available in Unix-like environment.

• I have installed ADMIXTOOLS using conda. It seems to be installed successfully. Detailed information could be found in the post link at the top of this page.

• I use Rstudio to maintain R packages. ADMIXR has been installed using Rstudio and data has been prepared according to the manual by @bodkan .

The related scripts are:

library(admixr)
library(tidyverse)

#load data by path 
prefix <- c("C:/writedata/bol2017/admixtools/bol2017")
snps_bol17 <- eigenstrat(prefix)

#chose some focal pops
pops <- c ("pop1","pop2","pop3")

# run a test
trialresult <- d(snps_bol17,W=pops,X="pop10",Y="pop6",Z="pop4")

All went well until the "run a test" step, the ERROR message:

#Error in file(con, "r") : cannot open connection
#additionally: Warning message:
#In file(con, "r") :
#cannot open file : 'C:\Users\NY_CAF\AppData\Local\Temp\Rtmpi6ginF/qpDstat__2090046701.log':No such file or directory

I went to this path and only two files were found. The .log file was NOT found.

I think "admixr" is a new and very cool R package. It will definitely become popular in population researchers, especially for those with little experiences for Unix environment, me included 😄. Hope to hear more and thanks again.

[bodkan]

Hi,

Thanks for reporting back.

Can you explain to me what you mean by

I use Ubuntu 18.04.1 as a subsystem

Are you running some dual-boot setup or are you running Ubuntu in some virtual machine/emulator within Windows? What is a "subsystem" in this context?

It's hard for me to understand what's going on without having access to this particular setup of a combination of Windows & Ubuntu & conda (none of the three I use).

I worry the only efficient solution might be to clone the admixr repository with this exact OS/conda setup, add a breakpoint (in RStudio) to the d() function and step through every line of code before one hits a suspicious piece of code.

Also, what happens when you run Sys.getenv("PATH") in RStudio?

[bodkan]

Closing for insufficient information. Anyone should feel free to reopen in case similar problems appear again.

[XiaXiaTianTian]

Hi, @bodkan

I run this in Windows using Rstudio. Loading admixr package.
My working directory is setwd("E:/tmp/04.final/admixtools") , and all needed files (.ind .snp .geno) are in this directory too.
But I got the same error:
#Error in file(con, "r") : cannot open connection #additionally: Warning message: #In file(con, "r") : #cannot open file : 'C:\Users\THINKC~1\AppData\Local\Temp\Rtmp6fOtxD/qpDstat__1072709677.log': No such file or directory

In C:\Users\THINKC~1\AppData\Local\Temp\Rtmp6fOtxD/, there are files as follows,

I follow your suggestion, run Sys.getenv("PATH"), got like this,
[1] "D:\\rstudio\\R-4.0.5\\bin\\x64;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;D:\\Xshell\\;D:\\mega\\MEGA11;D:\\xmanager\\;D:\\winscp\\WinSCP\\;C:\\Users\\ThinkCentre\\AppData\\Local\\Microsoft\\WindowsApps;D:\\Bandzip\\"

Great thanks for any possible suggestions.

Best,
Xia

[bodkan]

Hello Xia,

Thanks for adding more information to this issue. I have never progressed with Windows support, probably because ADMIXTOOLS itself is officially supported on macOS and Linux (not Windows, AFAIK) and I guess many users who are running Windows are doing analyses on computing clusters (which are almost exclusively Linux, I'd say). It's nice to have more information.

I have a couple of follow-up questions before we can proceed:

1. Could you please paste an example script which fails? I.e. a minimal possible script that gives you error (i.e. starting a line with library(admixr) all the way to the command that fails (I suspect it's d(...) in your case). I need to see the entire script, but as small as possible so that the error is immediately apparent. Just the error itself is unfortunately not sufficient.

2. What do you get when you run library(admixr) in your R console? Do you see any message from R?

3. Related question: what do you get when you run Sys.which("qpDstat")?

4. Another related question: how did you install ADMIXTOOLS? I don't think the Reich lab provides Windows binaries?

Cheers,
Martin

[XiaXiaTianTian]

Dear Martin,

I found ADMIXTOOLS version 7.0.2 6/22/21 (for Linux and Mac). But admixr is R package like other packages that can run on Rstudio in Windows, right? Are ADMIXTOOLS and admixr independent?

I followed your questions to check again. It showed warning message when run library(admixr) and it surely went error when run d(...). Additionaly, I did not install ADMIXTOOLS in windows or Rstuidio in Windows.

Here is some information.
`> library(admixr)
ADMIXTOOLS programs required by admixr are not in your $PATH variable.
At the very least, the following programs should be available:
qp3Pop, qpDstat, qpF4ratio, qpAdm, qpWave
Make sure to modify the $PATH variable in your .Renviron file so
that it points to the directory containing the ADMIXTOOLS programs.

Sys.which("qpDstat")
qpDstat
""
.libPaths()
[1] "D:/rstudio/R-4.0.5/library"
installed.packages()[,c('Package','Version','LibPath')]
Package Version LibPath
admixr "admixr" "0.9.1" "D:/rstudio/R-4.0.5/library"
which qpDstat
error: unexpected symbol in "which qpDstat"
`

When I continue to run command, it shows like following.
`> setwd("E:/tmp/04.final/admixtools/test")

prefix <- "./test"
snps <- eigenstrat(prefix)
list.files(path = dirname(prefix), pattern = basename(prefix), full.names = TRUE)
[1] "./test.geno" "./test.ind" "./test.map" "./test.ped" "./test.R" "./test.snp"
pops <- c("o1", "e", "m", "a1", "o1", "a2" , "o2" , "a")
result <- d(W = pops , X = "a1" , Y = "m", Z = "o1", data = snps)
#Error in file(con, "r") : cannot open connection
#additionally: Warning message: #In file(con, "r") : #cannot open file : 'C:\Users\THINKC~1\AppData\Local\Temp\Rtmp6fOtxD/qpDstat__1072709677.log': No such file or directory
`

Best,
Xia

[bodkan]

Hello again Xia,

I'm afraid I have some bad news. Also, thanks to the information you've provided, I finally understand the issue that the user above had back then.

I found ADMIXTOOLS version 7.0.2 6/22/21 (for Linux and Mac). But admixr is R package like other packages that can run on Rstudio in Windows, right? Are ADMIXTOOLS and admixr independent?

Yes, admixr and ADMIXTOOLS are two different pieces of software. Briefly, ADMIXTOOLS is a collection of command-line programs maintained by the Reich group. These are officially distributed for macOS and for Linux (AFAIK there's no official Windows support). The admixr R package that I'm developing serves as an R interface connecting to the underlying ADMIXTOOLS program.

Without ADMIXTOOLS present on the system, admixr can never work. This is why you see this message when you first load it:

> library(admixr)
ADMIXTOOLS programs required by admixr are not in your $PATH variable.
At the very least, the following programs should be available:
qp3Pop, qpDstat, qpF4ratio, qpAdm, qpWave
Make sure to modify the $PATH variable in your .Renviron file so
that it points to the directory containing the ADMIXTOOLS programs.

I know there is one way to run Linux subsystem on Windows called WSL. I suggest you give this a shot. Because this starts essentially a full-featured Linux environment under Windows, you should be able to install ADMIXTOOLS based on instructions from its authors. If you run into issues, I'm sure you can ask them for help.

Once you get ADMIXTOOLS itself running, please feel free to let me know here and I'd be more than happy to help you connect admixr from R to ADMIXTOOLS programs running on your system.

[jahnringge]

Hello bodkan! has Xia asked you more about connecting admixr and ADMIXTOOLS?

I am trying to run admixr through windows R. I have installed the Linux subsystem (runs ubuntu) WSL on windows and have used conda / bioconda to install ADMIXTOOLS. I confirm that admixtools is accessible through WSL.

I am running into issues connecting R to admixtools. The previous error that other users also encountered in this thread :
cannot open file 'C:\Users\jahn\AppData\Local\Temp\RtmpolV1nj/qpDstat__458398317.log': No such file or directory

What I have tried already:
Sys.setenv("PATH"="//wsl.localhost/Ubuntu/home/jahnringger/miniconda3/pkgs/admixtools-7.0.2-h2469040_2/bin/")

and variations of this. Additionally, when I search 'which qpDstat' on the R terminal, the output is 'no qpDstat in (/c/Users/jahn {list of environments}).'

Do you know of any way to access admixtools through R? Thanks in advance for the help

[bodkan]

Hi,

I'm afraid I don't remember the exact background to this issue. But I can reopen it to see if we can figure it out. To my knowledge, there aren't many ADMIXTOOLS users on Windows so the feedback (or possibility to debug issues) has been a bit tricky.

I don't have access to a Windows machine and don't really know WSL but the first thing to make sure that works (before we even bring admixr to the picture) is to make sure R itself can find the qpDstat et al. binaries.

For instance, on my machine, I get this:

> Sys.getenv("PATH")
[1] "[... stuff ...]:/Users/mp/.my_local/AdmixTools/bin: [... more stuff ...]"

And then I can verify this:

> Sys.which("qpDstat")
                                     qpDstat 
"/Users/mp/.my_local/AdmixTools/bin/qpDstat"

So if R can see qpDstat et al. in the $PATH, then admixr can also find it.

Can you test this? Before Sys.which("qpDstat") gives a positive result, admixr can't really do its job.

[bodkan]

I have found a (very suboptimal and slow) way to run Windows in a virtual machine on my Mac but setting up WSL, conda, ADMIXTOOLS from conda, etc. would take me a while as I don't know a thing about Windows.

I'm currently swamped with other bugs and issues in my other software, but if you don't find a way to do the Sys.which() thing above, I could poke around a bit later.

[jahnringge]

Hi bodkan,
thank you for your fast reply! Running

> Sys.which("qpDstat")
""

returns nothing, so I'm guessing that windows R has an issue with the $PATH.

I have since installed R on WSL and am running it through the linux terminal. This works well and I am able to work with admixr on my data.

However this means that I have to continuously move my data files from the windows environment (where I do most of my data analysis) to the linux environment. This is not very effective in the long run...

The problem is not urgent, but if you find the time to work on the Sys.which() issue I would appreciate the help, as my solution to the issue of admixr/ADMIXTOOLS in windows was.... not to use windows :)

[bodkan]

OK, yes. This:

> Sys.which("qpDstat")
""

means that it's a problem with finding the WSL ADMIXTOOLS installation in your Windows system. I think it goes even beyond R, to be honest.

My suggestion would be to google around the topic of "access path to a wsl binary on windows". Something like this might be useful? Again, I don't know Windows or WSL, so can't vouch for anything at this point.

There must be a way to access files installed via WSL from Windows. Like, even in the Windows command prompt. If you find how to refer to them, then you have your answer.

Also, perhaps I'm revealing a bit too much about my ignorance but I would normally write this:

Sys.setenv("PATH"="<path>")

as this:

Sys.setenv(PATH="<path>")

No idea if this makes any difference though.