HAIL#
In this notebook, we used Hail to calculate Polygenic Risk Scores (PRS). Hail does not calculate new betas but instead uses the existing weights from the GWAS and applies a custom formula for the calculation.
Possible issues#
Computation Time: Kindly note that HAIL is computationally expensive because it requires data in a specific format (.mt / matrix format), and then it loads the data for the computation.
We followed this tutorial to calculate the PRS:
https://nbviewer.org/github/ddbj/imputation-server-wf/blob/main/Notebooks/hail-prs-tutorial.ipynb
Basic Process#
Read the genotype data.
Convert it to VCF.
Use Beagle to convert the data to Beagle format.
Convert the Beagle format to Hail format.
Pass the data to Hail, GWAS, and genotype.
Calculate PRS using Hail.
Genotype Data Processing#
Convert genotype data to VCF format.
Hail requires data inbeagle.vcf.gzformat. The first step is to convert thebed,bim, andfamfiles to VCF format. A simple approach is to extract genotype data for each chromosome and then convert the data to VCF.plink --bfile traindirec/newtrainfilename.clumped.pruned --chr 22 --make-bed --out traindirec/newtrainfilename.clumped.pruned.22 plink --bfile traindirec/newtrainfilename.clumped.pruned.22 --chr 22 --recode vcf --out traindirec/hail.train.22
Download the phased reference panel.
To run the following commands, download the phased reference panel from 1000 Genomes and place it in the current working directory:
1000 Genomes Reference Panelwget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr1.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr2.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr3.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr4.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr5.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr7.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr8.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr9.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr10.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr11.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr12.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr13.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr14.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr15.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr16.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr17.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr18.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr19.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr20.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr21.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz wget https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
Alternatively, download the reference panel from Hail.
Reference panels:
Hail Reference PanelFiles (2.6 GB):
test-data.GRCh37.vcf.gz (1.3 GB,
md5:aff8bca4689cc70f6dbc1c3296590458)test-data.GRCh38.vcf.gz (1.3 GB,
md5:d28a741e820444ca926f7b0d5ac2e196)
Download genetic distances from the Beagle website.
Beagle Genetic MapsDownload Beagle.
Beagle documentation:
Beagle DocumentationBeagle download page:
Download Beagle
Run the Beagle command.
Run the following command to perform phasing and imputation:java -Xmx50g -jar beagle gt=traindirec/hail.train.22.vcf ref=ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr22.GRCh37.map out=traindirec/beagle.hail.train.22
Follow the remaining process.
The rest of the process is straightforward and can be followed from this GitHub repository:
PRS on Hail GitHub Repository
GWAS file processing for HAIL for Binary Phenotypes.#
When the effect size relates to disease risk and is thus given as an odds ratio (OR) rather than BETA (for continuous traits), the PRS is computed as a product of ORs. To simplify this calculation, take the natural logarithm of the OR so that the PRS can be computed using summation instead.
import os
import pandas as pd
import numpy as np
import sys
#filedirec = sys.argv[1]
filedirec = "SampleData1"
#filedirec = "asthma_19"
#filedirec = "migraine_0"
def check_phenotype_is_binary_or_continous(filedirec):
# Read the processed quality controlled file for a phenotype
df = pd.read_csv(filedirec+os.sep+filedirec+'_QC.fam',sep="\s+",header=None)
column_values = df[5].unique()
if len(set(column_values)) == 2:
return "Binary"
else:
return "Continous"
# Read the GWAS file.
GWAS = filedirec + os.sep + filedirec+".gz"
df = pd.read_csv(GWAS,compression= "gzip",sep="\s+")
if "BETA" in df.columns.to_list():
# For Continous Phenotype.
df = df[['CHR', 'BP', 'SNP', 'A1', 'A2', 'N', 'SE', 'P', 'BETA', 'INFO', 'MAF']]
else:
df["BETA"] = np.log(df["OR"])
df = df[['CHR', 'BP', 'SNP', 'A1', 'A2', 'N', 'SE', 'P', 'BETA', 'INFO', 'MAF']]
df = df.rename(columns={
'CHR': 'chr_name',
'BP': 'chr_position',
'A1': 'effect_allele',
'A2': 'other_allele',
'BETA': 'effect_weight'
})
# Selecting the relevant columns
df = df[['chr_name', 'chr_position', 'effect_allele', 'other_allele', 'effect_weight']]
# Remove duplicates based on 'chr_name' and 'chr_position'
print("Length of DataFrame!",len(df))
df = df.drop_duplicates(subset=['chr_name', 'chr_position', 'effect_allele', 'other_allele'])
print("Length of DataFrame!",len(df))
df.to_csv(filedirec + os.sep +"Hail.txt",sep="\t",index=False)
print(df.head().to_markdown())
print("Length of DataFrame!",len(df))
Length of DataFrame! 499617
Length of DataFrame! 499617
| | chr_name | chr_position | effect_allele | other_allele | effect_weight |
|---:|-----------:|---------------:|:----------------|:---------------|----------------:|
| 0 | 1 | 756604 | A | G | -0.00211532 |
| 1 | 1 | 768448 | A | G | 0.00068708 |
| 2 | 1 | 779322 | G | A | -0.00239932 |
| 3 | 1 | 801536 | G | T | 0.00203363 |
| 4 | 1 | 808631 | G | A | 0.00130747 |
Length of DataFrame! 499617
Define Hyperparameters#
Define hyperparameters to be optimized and set initial values.
Extract Valid SNPs from Clumped File#
For Windows, download gwak, and for Linux, the awk command is sufficient. For Windows, GWAK is required. You can download it from here. Get it and place it in the same directory.
Execution Path#
At this stage, we have the genotype training data newtrainfilename = "train_data.QC" and genotype test data newtestfilename = "test_data.QC".
We modified the following variables:
filedirec = "SampleData1"orfiledirec = sys.argv[1]foldnumber = "0"orfoldnumber = sys.argv[2]for HPC.
Only these two variables can be modified to execute the code for specific data and specific folds. Though the code can be executed separately for each fold on HPC and separately for each dataset, it is recommended to execute it for multiple diseases and one fold at a time. Here’s the corrected text in Markdown format:
P-values#
PRS calculation relies on P-values. SNPs with low P-values, indicating a high degree of association with a specific trait, are considered for calculation.
You can modify the code below to consider a specific set of P-values and save the file in the same format.
We considered the following parameters:
Minimum P-value:
1e-10Maximum P-value:
1.0Minimum exponent:
10(Minimum P-value in exponent)Number of intervals:
100(Number of intervals to be considered)
The code generates an array of logarithmically spaced P-values:
import numpy as np
import os
minimumpvalue = 10 # Minimum exponent for P-values
numberofintervals = 100 # Number of intervals to be considered
allpvalues = np.logspace(-minimumpvalue, 0, numberofintervals, endpoint=True) # Generating an array of logarithmically spaced P-values
print("Minimum P-value:", allpvalues[0])
print("Maximum P-value:", allpvalues[-1])
count = 1
with open(os.path.join(folddirec, 'range_list'), 'w') as file:
for value in allpvalues:
file.write(f'pv_{value} 0 {value}\n') # Writing range information to the 'range_list' file
count += 1
pvaluefile = os.path.join(folddirec, 'range_list')
In this code:
minimumpvaluedefines the minimum exponent for P-values.numberofintervalsspecifies how many intervals to consider.allpvaluesgenerates an array of P-values spaced logarithmically.The script writes these P-values to a file named
range_listin the specified directory.
from operator import index
import pandas as pd
import numpy as np
import os
import subprocess
import sys
import pandas as pd
import statsmodels.api as sm
import pandas as pd
from sklearn.metrics import roc_auc_score, confusion_matrix
from statsmodels.stats.contingency_tables import mcnemar
def create_directory(directory):
"""Function to create a directory if it doesn't exist."""
if not os.path.exists(directory): # Checking if the directory doesn't exist
os.makedirs(directory) # Creating the directory if it doesn't exist
return directory # Returning the created or existing directory
#foldnumber = sys.argv[1]
foldnumber = "0" # Setting 'foldnumber' to "0"
folddirec = filedirec + os.sep + "Fold_" + foldnumber # Creating a directory path for the specific fold
trainfilename = "train_data" # Setting the name of the training data file
newtrainfilename = "train_data.QC" # Setting the name of the new training data file
testfilename = "test_data" # Setting the name of the test data file
newtestfilename = "test_data.QC" # Setting the name of the new test data file
# Number of PCA to be included as a covariate.
numberofpca = ["6"] # Setting the number of PCA components to be included
# Clumping parameters.
clump_p1 = [1] # List containing clump parameter 'p1'
clump_r2 = [0.1] # List containing clump parameter 'r2'
clump_kb = [200] # List containing clump parameter 'kb'
# Pruning parameters.
p_window_size = [200] # List containing pruning parameter 'window_size'
p_slide_size = [50] # List containing pruning parameter 'slide_size'
p_LD_threshold = [0.25] # List containing pruning parameter 'LD_threshold'
# Kindly note that the number of p-values to be considered varies, and the actual p-value depends on the dataset as well.
# We will specify the range list here.
minimumpvalue = 10 # Minimum p-value in exponent
numberofintervals = 20 # Number of intervals to be considered
allpvalues = np.logspace(-minimumpvalue, 0, numberofintervals, endpoint=True) # Generating an array of logarithmically spaced p-values
count = 1
with open(folddirec + os.sep + 'range_list', 'w') as file:
for value in allpvalues:
file.write(f'pv_{value} 0 {value}\n') # Writing range information to the 'range_list' file
count = count + 1
pvaluefile = folddirec + os.sep + 'range_list'
# Initializing an empty DataFrame with specified column names
prs_result = pd.DataFrame(columns=["clump_p1", "clump_r2", "clump_kb", "p_window_size", "p_slide_size", "p_LD_threshold",
"pvalue", "numberofpca","numberofvariants","Train_pure_prs", "Train_null_model", "Train_best_model",
"Test_pure_prs", "Test_null_model", "Test_best_model"])
Define Helper Functions#
Perform Clumping and Pruning
Calculate PCA Using Plink
Fit Binary Phenotype and Save Results
Fit Continuous Phenotype and Save Results
import os
import subprocess
import pandas as pd
import statsmodels.api as sm
from sklearn.metrics import explained_variance_score
def perform_clumping_and_pruning_on_individual_data(traindirec, newtrainfilename,numberofpca, p1_val, p2_val, p3_val, c1_val, c2_val, c3_val,Name,pvaluefile):
command = [
"./plink",
"--bfile", traindirec+os.sep+newtrainfilename,
"--maf",str(0.2),
"--geno",str(0.001),
"--hwe",str(0.00001),
"--indep-pairwise", p1_val, p2_val, p3_val,
"--out", traindirec+os.sep+trainfilename
]
subprocess.run(command)
# First perform pruning and then clumping and the pruning.
command = [
"./plink",
"--bfile", traindirec+os.sep+newtrainfilename,
"--clump-p1", c1_val,
"--extract", traindirec+os.sep+trainfilename+".prune.in",
"--clump-r2", c2_val,
"--clump-kb", c3_val,
"--clump", filedirec+os.sep+filedirec+".txt",
"--clump-snp-field", "SNP",
"--clump-field", "P",
"--out", traindirec+os.sep+trainfilename
]
subprocess.run(command)
# Extract the valid SNPs from th clumped file.
# For windows download gwak for linux awk commmand is sufficient.
### For windows require GWAK.
### https://sourceforge.net/projects/gnuwin32/
##3 Get it and place it in the same direc.
#os.system("gawk "+"\""+"NR!=1{print $3}"+"\" "+ traindirec+os.sep+trainfilename+".clumped > "+traindirec+os.sep+trainfilename+".valid.snp")
#print("gawk "+"\""+"NR!=1{print $3}"+"\" "+ traindirec+os.sep+trainfilename+".clumped > "+traindirec+os.sep+trainfilename+".valid.snp")
#Linux:
command = f"awk 'NR!=1{{print $3}}' {traindirec}{os.sep}{trainfilename}.clumped > {traindirec}{os.sep}{trainfilename}.valid.snp"
os.system(command)
command = [
"./plink",
"--make-bed",
"--bfile", traindirec+os.sep+newtrainfilename,
"--indep-pairwise", p1_val, p2_val, p3_val,
"--extract", traindirec+os.sep+trainfilename+".valid.snp",
"--out", traindirec+os.sep+newtrainfilename+".clumped.pruned"
]
subprocess.run(command)
command = [
"./plink",
"--make-bed",
"--bfile", traindirec+os.sep+testfilename,
"--indep-pairwise", p1_val, p2_val, p3_val,
"--extract", traindirec+os.sep+trainfilename+".valid.snp",
"--out", traindirec+os.sep+testfilename+".clumped.pruned"
]
subprocess.run(command)
def calculate_pca_for_traindata_testdata_for_clumped_pruned_snps(traindirec, newtrainfilename,p):
# Calculate the PRS for the test data using the same set of SNPs and also calculate the PCA.
# Also extract the PCA at this point.
# PCA are calculated afer clumping and pruining.
command = [
"./plink",
"--bfile", folddirec+os.sep+testfilename+".clumped.pruned",
# Select the final variants after clumping and pruning.
"--extract", traindirec+os.sep+trainfilename+".valid.snp",
"--pca", p,
"--out", folddirec+os.sep+testfilename
]
subprocess.run(command)
command = [
"./plink",
"--bfile", traindirec+os.sep+newtrainfilename+".clumped.pruned",
# Select the final variants after clumping and pruning.
"--extract", traindirec+os.sep+trainfilename+".valid.snp",
"--pca", p,
"--out", traindirec+os.sep+trainfilename
]
subprocess.run(command)
# This function fit the binary model on the PRS.
def fit_binary_phenotype_on_PRS(traindirec, newtrainfilename,p, p1_val, p2_val, p3_val, c1_val, c2_val, c3_val,Name,pvaluefile):
threshold_values = allpvalues
# Merge the covariates, pca and phenotypes.
tempphenotype_train = pd.read_table(traindirec+os.sep+newtrainfilename+".clumped.pruned"+".fam", sep="\s+",header=None)
phenotype_train = pd.DataFrame()
phenotype_train["Phenotype"] = tempphenotype_train[5].values
pcs_train = pd.read_table(traindirec+os.sep+trainfilename+".eigenvec", sep="\s+",header=None, names=["FID", "IID"] + [f"PC{str(i)}" for i in range(1, int(p)+1)])
covariate_train = pd.read_table(traindirec+os.sep+trainfilename+".cov",sep="\s+")
covariate_train.fillna(0, inplace=True)
covariate_train = covariate_train[covariate_train["FID"].isin(pcs_train["FID"].values) & covariate_train["IID"].isin(pcs_train["IID"].values)]
covariate_train['FID'] = covariate_train['FID'].astype(str)
pcs_train['FID'] = pcs_train['FID'].astype(str)
covariate_train['IID'] = covariate_train['IID'].astype(str)
pcs_train['IID'] = pcs_train['IID'].astype(str)
covandpcs_train = pd.merge(covariate_train, pcs_train, on=["FID","IID"])
covandpcs_train.fillna(0, inplace=True)
## Scale the covariates!
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import explained_variance_score
scaler = MinMaxScaler()
normalized_values_train = scaler.fit_transform(covandpcs_train.iloc[:, 2:])
#covandpcs_train.iloc[:, 2:] = normalized_values_test
tempphenotype_test = pd.read_table(traindirec+os.sep+testfilename+".clumped.pruned"+".fam", sep="\s+",header=None)
phenotype_test= pd.DataFrame()
phenotype_test["Phenotype"] = tempphenotype_test[5].values
pcs_test = pd.read_table(traindirec+os.sep+testfilename+".eigenvec", sep="\s+",header=None, names=["FID", "IID"] + [f"PC{str(i)}" for i in range(1, int(p)+1)])
covariate_test = pd.read_table(traindirec+os.sep+testfilename+".cov",sep="\s+")
covariate_test.fillna(0, inplace=True)
covariate_test = covariate_test[covariate_test["FID"].isin(pcs_test["FID"].values) & covariate_test["IID"].isin(pcs_test["IID"].values)]
covariate_test['FID'] = covariate_test['FID'].astype(str)
pcs_test['FID'] = pcs_test['FID'].astype(str)
covariate_test['IID'] = covariate_test['IID'].astype(str)
pcs_test['IID'] = pcs_test['IID'].astype(str)
covandpcs_test = pd.merge(covariate_test, pcs_test, on=["FID","IID"])
covandpcs_test.fillna(0, inplace=True)
normalized_values_test = scaler.transform(covandpcs_test.iloc[:, 2:])
#covandpcs_test.iloc[:, 2:] = normalized_values_test
tempalphas = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
l1weights = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
tempalphas = [0.1]
l1weights = [0.1]
phenotype_train["Phenotype"] = phenotype_train["Phenotype"].replace({1: 0, 2: 1})
phenotype_test["Phenotype"] = phenotype_test["Phenotype"].replace({1: 0, 2: 1})
for tempalpha in tempalphas:
for l1weight in l1weights:
try:
null_model = sm.Logit(phenotype_train["Phenotype"], sm.add_constant(covandpcs_train.iloc[:, 2:])).fit_regularized(alpha=tempalpha, L1_wt=l1weight)
#null_model = sm.Logit(phenotype_train["Phenotype"], sm.add_constant(covandpcs_train.iloc[:, 2:])).fit()
except:
print("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
continue
train_null_predicted = null_model.predict(sm.add_constant(covandpcs_train.iloc[:, 2:]))
from sklearn.metrics import roc_auc_score, confusion_matrix
from sklearn.metrics import r2_score
test_null_predicted = null_model.predict(sm.add_constant(covandpcs_test.iloc[:, 2:]))
global prs_result
prs_train = pd.read_table(traindirec+os.sep+Name+os.sep+"train_hail_prs.csv",sep=",",index_col=0)
print(prs_train)
prs_train[['FID', 'IID']] = prs_train['subjectID'].str.split('_', expand=True)
# Drop the original 'Combined' column if it's no longer needed
prs_train = prs_train.drop(columns=['subjectID'])
prs_train.rename(columns={'PGS002724': 'SCORE'}, inplace=True)
prs_train['FID'] = prs_train['FID'].astype(str)
prs_train['IID'] = prs_train['IID'].astype(str)
prs_test = pd.read_table(traindirec+os.sep+Name+os.sep+"test_hail_prs.csv",sep=",",index_col=0)
print(prs_test)
prs_test[['FID', 'IID']] = prs_test['subjectID'].str.split('_', expand=True)
# Drop the original 'Combined' column if it's no longer needed
prs_test = prs_test.drop(columns=['subjectID'])
prs_test.rename(columns={'PGS002724': 'SCORE'}, inplace=True)
prs_test['FID'] = prs_test['FID'].astype(str)
prs_test['IID'] = prs_test['IID'].astype(str)
pheno_prs_train = pd.merge(covandpcs_train, prs_train, on=["FID", "IID"])
pheno_prs_test = pd.merge(covandpcs_test, prs_test, on=["FID", "IID"])
try:
model = sm.Logit(phenotype_train["Phenotype"], sm.add_constant(pheno_prs_train.iloc[:, 2:])).fit_regularized(alpha=tempalpha, L1_wt=l1weight)
#model = sm.Logit(phenotype_train["Phenotype"], sm.add_constant(pheno_prs_train.iloc[:, 2:])).fit()
except:
continue
train_best_predicted = model.predict(sm.add_constant(pheno_prs_train.iloc[:, 2:]))
test_best_predicted = model.predict(sm.add_constant(pheno_prs_test.iloc[:, 2:]))
from sklearn.metrics import roc_auc_score, confusion_matrix
prs_result = prs_result._append({
"clump_p1": c1_val,
"clump_r2": c2_val,
"clump_kb": c3_val,
"p_window_size": p1_val,
"p_slide_size": p2_val,
"p_LD_threshold": p3_val,
#"pvalue": i,
"numberofpca":p,
"tempalpha":str(tempalpha),
"l1weight":str(l1weight),
"Train_pure_prs":roc_auc_score(phenotype_train["Phenotype"].values,prs_train['SCORE'].values),
"Train_null_model":roc_auc_score(phenotype_train["Phenotype"].values,train_null_predicted.values),
"Train_best_model":roc_auc_score(phenotype_train["Phenotype"].values,train_best_predicted.values),
"Test_pure_prs":roc_auc_score(phenotype_test["Phenotype"].values,prs_test['SCORE'].values),
"Test_null_model":roc_auc_score(phenotype_test["Phenotype"].values,test_null_predicted.values),
"Test_best_model":roc_auc_score(phenotype_test["Phenotype"].values,test_best_predicted.values),
}, ignore_index=True)
prs_result.to_csv(traindirec+os.sep+Name+os.sep+"Results.csv",index=False)
# This function fit the binary model on the PRS.
def fit_continous_phenotype_on_PRS(traindirec, newtrainfilename,p, p1_val, p2_val, p3_val, c1_val, c2_val, c3_val,Name,pvaluefile):
threshold_values = allpvalues
# Merge the covariates, pca and phenotypes.
tempphenotype_train = pd.read_table(traindirec+os.sep+newtrainfilename+".clumped.pruned"+".fam", sep="\s+",header=None)
phenotype_train = pd.DataFrame()
phenotype_train["Phenotype"] = tempphenotype_train[5].values
pcs_train = pd.read_table(traindirec+os.sep+trainfilename+".eigenvec", sep="\s+",header=None, names=["FID", "IID"] + [f"PC{str(i)}" for i in range(1, int(p)+1)])
covariate_train = pd.read_table(traindirec+os.sep+trainfilename+".cov",sep="\s+")
covariate_train.fillna(0, inplace=True)
covariate_train = covariate_train[covariate_train["FID"].isin(pcs_train["FID"].values) & covariate_train["IID"].isin(pcs_train["IID"].values)]
covariate_train['FID'] = covariate_train['FID'].astype(str)
pcs_train['FID'] = pcs_train['FID'].astype(str)
covariate_train['IID'] = covariate_train['IID'].astype(str)
pcs_train['IID'] = pcs_train['IID'].astype(str)
covandpcs_train = pd.merge(covariate_train, pcs_train, on=["FID","IID"])
covandpcs_train.fillna(0, inplace=True)
## Scale the covariates!
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import explained_variance_score
scaler = MinMaxScaler()
normalized_values_train = scaler.fit_transform(covandpcs_train.iloc[:, 2:])
#covandpcs_train.iloc[:, 2:] = normalized_values_test
tempphenotype_test = pd.read_table(traindirec+os.sep+testfilename+".clumped.pruned"+".fam", sep="\s+",header=None)
phenotype_test= pd.DataFrame()
phenotype_test["Phenotype"] = tempphenotype_test[5].values
pcs_test = pd.read_table(traindirec+os.sep+testfilename+".eigenvec", sep="\s+",header=None, names=["FID", "IID"] + [f"PC{str(i)}" for i in range(1, int(p)+1)])
covariate_test = pd.read_table(traindirec+os.sep+testfilename+".cov",sep="\s+")
covariate_test.fillna(0, inplace=True)
covariate_test = covariate_test[covariate_test["FID"].isin(pcs_test["FID"].values) & covariate_test["IID"].isin(pcs_test["IID"].values)]
covariate_test['FID'] = covariate_test['FID'].astype(str)
pcs_test['FID'] = pcs_test['FID'].astype(str)
covariate_test['IID'] = covariate_test['IID'].astype(str)
pcs_test['IID'] = pcs_test['IID'].astype(str)
covandpcs_test = pd.merge(covariate_test, pcs_test, on=["FID","IID"])
covandpcs_test.fillna(0, inplace=True)
normalized_values_test = scaler.transform(covandpcs_test.iloc[:, 2:])
#covandpcs_test.iloc[:, 2:] = normalized_values_test
tempalphas = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
l1weights = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
tempalphas = [0.1]
l1weights = [0.1]
#phenotype_train["Phenotype"] = phenotype_train["Phenotype"].replace({1: 0, 2: 1})
#phenotype_test["Phenotype"] = phenotype_test["Phenotype"].replace({1: 0, 2: 1})
for tempalpha in tempalphas:
for l1weight in l1weights:
try:
#null_model = sm.OLS(phenotype_train["Phenotype"], sm.add_constant(covandpcs_train.iloc[:, 2:])).fit_regularized(alpha=tempalpha, L1_wt=l1weight)
null_model = sm.OLS(phenotype_train["Phenotype"], sm.add_constant(covandpcs_train.iloc[:, 2:])).fit()
#null_model = sm.OLS(phenotype_train["Phenotype"], sm.add_constant(covandpcs_train.iloc[:, 2:])).fit()
except:
print("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
continue
train_null_predicted = null_model.predict(sm.add_constant(covandpcs_train.iloc[:, 2:]))
from sklearn.metrics import roc_auc_score, confusion_matrix
from sklearn.metrics import r2_score
test_null_predicted = null_model.predict(sm.add_constant(covandpcs_test.iloc[:, 2:]))
global prs_result
prs_train = pd.read_table(traindirec+os.sep+Name+os.sep+"train_hail_prs.csv",sep=",",index_col=0)
print(prs_train)
prs_train[['FID', 'IID']] = prs_train['subjectID'].str.split('_', expand=True)
# Drop the original 'Combined' column if it's no longer needed
prs_train = prs_train.drop(columns=['subjectID'])
prs_train.rename(columns={'PGS002724': 'SCORE'}, inplace=True)
prs_train['FID'] = prs_train['FID'].astype(str)
prs_train['IID'] = prs_train['IID'].astype(str)
prs_test = pd.read_table(traindirec+os.sep+Name+os.sep+"test_hail_prs.csv",sep=",",index_col=0)
prs_test[['FID', 'IID']] = prs_test['subjectID'].str.split('_', expand=True)
# Drop the original 'Combined' column if it's no longer needed
prs_test = prs_test.drop(columns=['subjectID'])
prs_test.rename(columns={'PGS002724': 'SCORE'}, inplace=True)
print(prs_test)
prs_test['FID'] = prs_test['FID'].astype(str)
prs_test['IID'] = prs_test['IID'].astype(str)
pheno_prs_train = pd.merge(covandpcs_train, prs_train, on=["FID", "IID"])
pheno_prs_test = pd.merge(covandpcs_test, prs_test, on=["FID", "IID"])
print(pheno_prs_train)
try:
#model = sm.OLS(phenotype_train["Phenotype"], sm.add_constant(pheno_prs_train.iloc[:, 2:])).fit_regularized(alpha=tempalpha, L1_wt=l1weight)
model = sm.OLS(phenotype_train["Phenotype"], sm.add_constant(pheno_prs_train.iloc[:, 2:])).fit()
except:
print("Model did not fit")
continue
train_best_predicted = model.predict(sm.add_constant(pheno_prs_train.iloc[:, 2:]))
test_best_predicted = model.predict(sm.add_constant(pheno_prs_test.iloc[:, 2:]))
from sklearn.metrics import roc_auc_score, confusion_matrix
prs_result = prs_result._append({
"clump_p1": c1_val,
"clump_r2": c2_val,
"clump_kb": c3_val,
"p_window_size": p1_val,
"p_slide_size": p2_val,
"p_LD_threshold": p3_val,
"numberofpca":p,
"tempalpha":str(tempalpha),
"l1weight":str(l1weight),
"Train_pure_prs":explained_variance_score(phenotype_train["Phenotype"],prs_train['SCORE'].values),
"Train_null_model":explained_variance_score(phenotype_train["Phenotype"],train_null_predicted),
"Train_best_model":explained_variance_score(phenotype_train["Phenotype"],train_best_predicted),
"Test_pure_prs":explained_variance_score(phenotype_test["Phenotype"],prs_test['SCORE'].values),
"Test_null_model":explained_variance_score(phenotype_test["Phenotype"],test_null_predicted),
"Test_best_model":explained_variance_score(phenotype_test["Phenotype"],test_best_predicted),
}, ignore_index=True)
print(prs_result)
prs_result.to_csv(traindirec+os.sep+Name+os.sep+"Results.csv",index=False)
return
Execute HAIL#
import hail as hl
import shutil
import os
# Define a global variable to store results
prs_result = pd.DataFrame()
def transform_hail_data(traindirec, newtrainfilename,p, p1_val, p2_val, p3_val, c1_val, c2_val, c3_val,Name,pvaluefile):
### First perform clumping on the file and save the clumpled file.
perform_clumping_and_pruning_on_individual_data(traindirec, newtrainfilename,p, p1_val, p2_val, p3_val, c1_val, c2_val, c3_val,Name,pvaluefile)
#newtrainfilename = newtrainfilename+".clumped.pruned"
#testfilename = testfilename+".clumped.pruned"
#clupmedfile = traindirec+os.sep+newtrainfilename+".clump"
#prunedfile = traindirec+os.sep+newtrainfilename+".clumped.pruned"
# Also extract the PCA at this point for both test and training data.
calculate_pca_for_traindata_testdata_for_clumped_pruned_snps(traindirec, newtrainfilename,p)
# Delete the files generated in the previous iteration
# Loop through each chromosome and delete the corresponding files
import time
def remove_path(path):
try:
if os.path.isdir(path):
shutil.rmtree(path) # Remove directory
print(f'Deleted directory: {path}')
else:
os.remove(path) # Remove file
print(f'Deleted file: {path}')
except OSError as e:
print(f'Error deleting {path}: {e}')
print('Retrying...')
# Loop through each chromosome and delete the corresponding files and directories
for chromosome in range(1, 23):
train_path = os.path.join(traindirec, f'hail.train.{chromosome}.mt')
test_path = os.path.join(traindirec, f'hail.test.{chromosome}.mt')
# Delete train path if it exists
if os.path.exists(train_path):
remove_path(train_path)
pass
# Delete test path if it exists
if os.path.exists(test_path):
remove_path(test_path)
pass
#"""
# Here we will perform processing on the genotype data and convert it to specific format
# as required by HAIL.
# Read the genotype data
# Convert it to VCF
# Use beagle to convert the data to beagle format
# Convert the beagle format to Hail format.
for chromosome in range(1, 23):
# Plink command to split by chromosome
plink_command = [
"./plink",
"--bfile", traindirec+os.sep+newtrainfilename+".clumped.pruned",
"--chr", str(chromosome),
"--make-bed",
"--out", traindirec+os.sep+newtrainfilename+".clumped.pruned."+str(chromosome),
]
try:
subprocess.run(plink_command, check=True)
except:
pass
# Convert chromosomes to VCF format.
plink_command = [
"./plink",
"--bfile", traindirec+os.sep+newtrainfilename+".clumped.pruned."+str(chromosome),
"--chr", str(chromosome),
"--recode","vcf",
"--out", traindirec+os.sep+"hail.train."+str(chromosome),
]
try:
subprocess.run(plink_command)
except:
pass
# Run beagle and convert the data to beagle format.
#A “DR2” subfield with the estimated squared correlation between the estimated allele
#dose and the true allele dose
#• An “AF” subfield with the estimated alternate allele frequencies in the target samples
#• The “IMP” flag if the marker is imputed
beagle_command = [
"java",
"-Xmx50g",
"-jar", "beagle",
#"gp=true",
#"impute=false",
#"burnin=1",
#"window=100",
"gt="+traindirec+os.sep+"hail.train."+str(chromosome)+".vcf",
"ref="+"ALL.chr"+str(chromosome)+".phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz",
"map="+"plink.chr"+str(chromosome)+".GRCh37.map",
"out="+traindirec+os.sep+"beagle.hail.train."+str(chromosome)
]
# Execute the command
try:
subprocess.run(beagle_command)
print(" ".join(beagle_command))
except:
pass
# Convert beagle format to HAIL format.
try:
hl.import_vcf(traindirec+os.sep+"beagle.hail.train."+str(chromosome)+".vcf.gz", force_bgz=True).write(traindirec+os.sep+"hail.train."+str(chromosome)+".mt", overwrite=True)
except:
pass
# Read the genotype data
mt = ""
filecount = 0
for chromosome in range(1, 23):
file = os.path.join(traindirec, f"hail.train.{chromosome}.mt")
if os.path.exists(file): # Check if the file exists
if filecount == 0:
mt = hl.read_matrix_table(file)
print(mt.rows().show(5))
print(chromosome, file, mt.count(), mt.count())
filecount += 1 # Increment filecount after the first file is processed
else:
tmpmt = hl.read_matrix_table(file)
mt = mt.union_rows(tmpmt)
print(chromosome, file, tmpmt.count(), mt.count())
else:
print("File does not exist:", file)
print(mt.count())
mt.write(traindirec+os.sep+'HAILTRAIN.mt', overwrite=True)
mt = hl.read_matrix_table(traindirec+os.sep+'HAILTRAIN.mt')
print(mt.rows().show(5))
mt = mt.annotate_rows(variantID = (hl.str(mt.locus.contig) + ":" + hl.str(mt.locus.position)) )
# We skipped the cleaning process as the data is already clean.
#print(mt.rows().show(5))
#mt1 = mt.filter_rows(hl.len(mt.info.DR2) == 1)
#mtnot1 = mt.filter_rows(hl.len(mt.info.DR2) > 1)
#mt1_filt = mt1.filter_rows(mt1.info.DR2.first()>=0.3)
#print(mt1_filt.rows().show(5))
model_PGS002724 = hl.import_table(filedirec+os.sep+"Hail.txt", impute=True, force=True, comment='#')
model_PGS002724 = model_PGS002724.annotate(
variantID = hl.str(model_PGS002724.chr_name) + ":" + hl.str(model_PGS002724.chr_position)
)
model_PGS002724 = model_PGS002724.key_by('variantID')
mt_match_PGS002724 = mt.annotate_rows(**model_PGS002724[mt.variantID])
mt_match_PGS002724 = mt_match_PGS002724.filter_rows(hl.is_defined(mt_match_PGS002724.effect_weight))
flip_PGS002724 = hl.case().when(
(mt_match_PGS002724.effect_allele == mt_match_PGS002724.alleles[0])
& (mt_match_PGS002724.other_allele == mt_match_PGS002724.alleles[1]), True ).when(
(mt_match_PGS002724.effect_allele == mt_match_PGS002724.alleles[1])
& (mt_match_PGS002724.other_allele == mt_match_PGS002724.alleles[0]), False ).or_missing()
mt_match_PGS002724 = mt_match_PGS002724.annotate_rows(flip=flip_PGS002724)
prs_PGS002724 = hl.agg.sum(hl.float64(mt_match_PGS002724.effect_weight) *
hl.if_else( mt_match_PGS002724.flip,
2 - mt_match_PGS002724.DS.first(),
mt_match_PGS002724.DS.first()))
mt_match_PGS002724 = mt_match_PGS002724.annotate_cols(prs=prs_PGS002724)
mt_match_PGS002724.cols().export(traindirec+os.sep+Name+os.sep+'train_PRS.txt')
prs_PGS002724 = hl.import_table(traindirec+os.sep+Name+os.sep+'train_PRS.txt', impute=True, force=True)
prs_PGS002724 = prs_PGS002724.key_by('s')
prs_merge = prs_PGS002724.rename({'s':'subjectID', 'prs':'PGS002724'})
prs_merge_pandas = prs_merge.to_pandas()
print(prs_merge_pandas.head())
prs_merge_pandas.to_csv(traindirec+os.sep+Name+os.sep+"train_hail_prs.csv")
# Save the data.
prs_merge_pandas = pd.read_csv(traindirec+os.sep+Name+os.sep+"train_hail_prs.csv",index_col=0)
print(prs_merge_pandas.head())
#"""
#"""
# Repeat the process for test dataset.
# testfilename
for chromosome in range(1, 23):
# Plink command to split by chromosome
plink_command = [
"./plink",
"--bfile", traindirec+os.sep+testfilename+".clumped.pruned",
"--chr", str(chromosome),
"--make-bed",
"--out", traindirec+os.sep+testfilename+".clumped.pruned."+str(chromosome),
]
try:
subprocess.run(plink_command)
except:
pass
plink_command = [
"./plink",
"--bfile",traindirec+os.sep+testfilename+".clumped.pruned."+str(chromosome),
"--chr", str(chromosome),
"--recode","vcf",
"--out", traindirec+os.sep+"hail.test."+str(chromosome),
]
try:
subprocess.run(plink_command, check=True)
except:
pass
beagle_command = [
"java",
"-Xmx50g",
"-jar", "beagle",
#"gp=true",
#"burnin=1",
#"window=100",
"gt="+traindirec+os.sep+"hail.test."+str(chromosome)+".vcf",
"ref="+"ALL.chr"+str(chromosome)+".phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz",
"map="+"plink.chr"+str(chromosome)+".GRCh37.map",
"out="+traindirec+os.sep+"beagle.hail.test."+str(chromosome)
]
# Execute the command
try:
subprocess.run(beagle_command)
print(" ".join(beagle_command))
except:
pass
#raise
try:
hl.import_vcf(traindirec+os.sep+"beagle.hail.test."+str(chromosome)+".vcf.gz", force_bgz=True).write(traindirec+os.sep+"hail.test."+str(chromosome)+".mt", overwrite=True)
#raise
except:
pass
mt = ""
filecount = 0
for chromosome in range(1, 23):
file = os.path.join(traindirec, f"hail.test.{chromosome}.mt")
if os.path.exists(file): # Check if the file exists
if filecount == 0:
mt = hl.read_matrix_table(file)
print(mt.rows().show(5))
print(chromosome, file, mt.count(), mt.count())
filecount += 1 # Increment filecount after the first file is processed
else:
tmpmt = hl.read_matrix_table(file)
mt = mt.union_rows(tmpmt)
print(chromosome, file, tmpmt.count(), mt.count())
else:
print("File does not exist:", file)
print(mt.count())
mt.write(traindirec+os.sep+'HAILTEST.mt', overwrite=True)
#"""
#convert_data_in_hail_format()
#"""
mt = hl.read_matrix_table(traindirec+os.sep+'HAILTEST.mt')
print(mt.rows().show(5))
mt = mt.annotate_rows(variantID = (hl.str(mt.locus.contig) + ":" + hl.str(mt.locus.position)) )
#print(mt.rows().show(5))
#mt1 = mt.filter_rows(hl.len(mt.info.DR2) == 1)
#mtnot1 = mt.filter_rows(hl.len(mt.info.DR2) > 1)
#mt1_filt = mt1.filter_rows(mt1.info.DR2.first()>=0.3)
#print(mt1_filt.rows().show(5))
model_PGS002724 = hl.import_table(filedirec+os.sep+"Hail.txt", impute=True, force=True, comment='#')
model_PGS002724 = model_PGS002724.annotate(
variantID = hl.str(model_PGS002724.chr_name) + ":" + hl.str(model_PGS002724.chr_position)
)
model_PGS002724 = model_PGS002724.key_by('variantID')
mt_match_PGS002724 = mt.annotate_rows(**model_PGS002724[mt.variantID])
mt_match_PGS002724 = mt_match_PGS002724.filter_rows(hl.is_defined(mt_match_PGS002724.effect_weight))
flip_PGS002724 = hl.case().when(
(mt_match_PGS002724.effect_allele == mt_match_PGS002724.alleles[0])
& (mt_match_PGS002724.other_allele == mt_match_PGS002724.alleles[1]), True ).when(
(mt_match_PGS002724.effect_allele == mt_match_PGS002724.alleles[1])
& (mt_match_PGS002724.other_allele == mt_match_PGS002724.alleles[0]), False ).or_missing()
mt_match_PGS002724 = mt_match_PGS002724.annotate_rows(flip=flip_PGS002724)
prs_PGS002724 = hl.agg.sum(hl.float64(mt_match_PGS002724.effect_weight) *
hl.if_else( mt_match_PGS002724.flip,
2 - mt_match_PGS002724.DS.first(),
mt_match_PGS002724.DS.first()))
mt_match_PGS002724 = mt_match_PGS002724.annotate_cols(prs=prs_PGS002724)
mt_match_PGS002724.cols().export(traindirec+os.sep+Name+os.sep+'test_PRS.txt')
prs_PGS002724 = hl.import_table(traindirec+os.sep+Name+os.sep+'test_PRS.txt', impute=True, force=True)
prs_PGS002724 = prs_PGS002724.key_by('s')
prs_merge = prs_PGS002724.rename({'s':'subjectID', 'prs':'PGS002724'})
prs_merge_pandas = prs_merge.to_pandas()
print(prs_merge_pandas.head())
prs_merge_pandas.to_csv(traindirec+os.sep+Name+os.sep+"test_hail_prs.csv")
prs_merge_pandas = pd.read_csv(traindirec+os.sep+Name+os.sep+"test_hail_prs.csv",index_col=0)
print(prs_merge_pandas.head())
#"""
# Load the PCA and Load the Covariates for trainingdatafirst.
if check_phenotype_is_binary_or_continous(filedirec)=="Binary":
print("Binary Phenotype!")
fit_binary_phenotype_on_PRS(traindirec, newtrainfilename,p, p1_val, p2_val, p3_val, c1_val, c2_val, c3_val,Name,pvaluefile)
else:
print("Continous Phenotype!")
fit_continous_phenotype_on_PRS(traindirec, newtrainfilename,p, p1_val, p2_val, p3_val, c1_val, c2_val, c3_val,Name,pvaluefile)
result_directory = "HAIL"
# Nested loops to iterate over different parameter values
create_directory(folddirec+os.sep+result_directory)
for p1_val in p_window_size:
for p2_val in p_slide_size:
for p3_val in p_LD_threshold:
for c1_val in clump_p1:
for c2_val in clump_r2:
for c3_val in clump_kb:
for p in numberofpca:
transform_hail_data(folddirec, newtrainfilename, p, str(p1_val), str(p2_val), str(p3_val), str(c1_val), str(c2_val), str(c3_val), result_directory, pvaluefile)
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC
--geno 0.001
--hwe 1e-05
--indep-pairwise 200 50 0.25
--maf 0.2
--out SampleData1/Fold_0/train_data
63761 MB RAM detected; reserving 31880 MB for main workspace.
491952 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999894.
4954 variants removed due to missing genotype data (--geno).
--hwe: 0 variants removed due to Hardy-Weinberg exact test.
346375 variants removed due to minor allele threshold(s)
(--maf/--max-maf/--mac/--max-mac).
140623 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
Pruned 6339 variants from chromosome 1, leaving 4777.
Pruned 6277 variants from chromosome 2, leaving 4595.
Pruned 5191 variants from chromosome 3, leaving 3992.
Pruned 4996 variants from chromosome 4, leaving 3727.
Pruned 4537 variants from chromosome 5, leaving 3581.
Pruned 6868 variants from chromosome 6, leaving 3561.
Pruned 4379 variants from chromosome 7, leaving 3276.
Pruned 4071 variants from chromosome 8, leaving 2898.
Pruned 3287 variants from chromosome 9, leaving 2823.
Pruned 3822 variants from chromosome 10, leaving 3054.
Pruned 4158 variants from chromosome 11, leaving 2849.
Pruned 3544 variants from chromosome 12, leaving 2868.
Pruned 2560 variants from chromosome 13, leaving 2246.
Pruned 2424 variants from chromosome 14, leaving 1989.
Pruned 2615 variants from chromosome 15, leaving 2060.
Pruned 2867 variants from chromosome 16, leaving 2231.
Pruned 2782 variants from chromosome 17, leaving 2164.
Pruned 2223 variants from chromosome 18, leaving 2047.
Pruned 2593 variants from chromosome 19, leaving 1818.
Pruned 1980 variants from chromosome 20, leaving 1779.
Pruned 1254 variants from chromosome 21, leaving 992.
Pruned 1403 variants from chromosome 22, leaving 1126.
Pruning complete. 80170 of 140623 variants removed.
Marker lists written to SampleData1/Fold_0/train_data.prune.in and
SampleData1/Fold_0/train_data.prune.out .
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC
--clump SampleData1/SampleData1.txt
--clump-field P
--clump-kb 200
--clump-p1 1
--clump-r2 0.1
--clump-snp-field SNP
--extract SampleData1/Fold_0/train_data.prune.in
--out SampleData1/Fold_0/train_data
63761 MB RAM detected; reserving 31880 MB for main workspace.
491952 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
--extract: 60453 variants remaining.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is exactly 1.
60453 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--clump: 38646 clumps formed from 60453 top variants.
Results written to SampleData1/Fold_0/train_data.clumped .
Warning: 'rs3134762' is missing from the main dataset, and is a top variant.
Warning: 'rs3132505' is missing from the main dataset, and is a top variant.
Warning: 'rs3130424' is missing from the main dataset, and is a top variant.
439161 more top variant IDs missing; see log file.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC
--extract SampleData1/Fold_0/train_data.valid.snp
--indep-pairwise 200 50 0.25
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned
63761 MB RAM detected; reserving 31880 MB for main workspace.
491952 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
--extract: 77998 variants remaining.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99% done.
Total genotyping rate is 0.999939.
77998 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.fam ... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%done.
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
Pruned 2 variants from chromosome 1, leaving 6245.
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
42%
43%
44%
45%
46%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
95%
96%
97%
98%
99%
Pruned 2 variants from chromosome 2, leaving 5993.
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
29%
30%
31%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
48%
49%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
Pruned 1 variant from chromosome 3, leaving 5253.
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
22%
23%
25%
26%
28%
29%
30%
31%
32%
33%
34%
35%
36%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
49%
50%
51%
52%
53%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
68%
69%
70%
71%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
91%
92%
93%
94%
95%
96%
97%
98%
99%
Pruned 0 variants from chromosome 4, leaving 4966.
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
26%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
46%
47%
48%
49%
50%
51%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
Pruned 4 variants from chromosome 5, leaving 4812.
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
11%
12%
13%
14%
15%
16%
25%
34%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
48%
49%
50%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
Warning: At least 5268 duplicate IDs in --extract file.
Pruned 3 variants from chromosome 6, leaving 4414.
Pruned 4 variants from chromosome 7, leaving 4212.
Pruned 2 variants from chromosome 8, leaving 3866.
Pruned 1 variant from chromosome 9, leaving 3555.
Pruned 3 variants from chromosome 10, leaving 3949.
Pruned 1 variant from chromosome 11, leaving 3758.
Pruned 1 variant from chromosome 12, leaving 3715.
Pruned 2 variants from chromosome 13, leaving 2866.
Pruned 1 variant from chromosome 14, leaving 2590.
Pruned 2 variants from chromosome 15, leaving 2604.
Pruned 2 variants from chromosome 16, leaving 2870.
Pruned 0 variants from chromosome 17, leaving 2668.
Pruned 0 variants from chromosome 18, leaving 2603.
Pruned 0 variants from chromosome 19, leaving 2136.
Pruned 1 variant from chromosome 20, leaving 2239.
Pruned 0 variants from chromosome 21, leaving 1331.
Pruned 0 variants from chromosome 22, leaving 1321.
Pruning complete. 32 of 77998 variants removed.
Marker lists written to
SampleData1/Fold_0/train_data.QC.clumped.pruned.prune.in and
SampleData1/Fold_0/train_data.QC.clumped.pruned.prune.out .
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data
--extract SampleData1/Fold_0/train_data.valid.snp
--indep-pairwise 200 50 0.25
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned
63761 MB RAM detected; reserving 31880 MB for main workspace.
551892 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
--extract: 77998 variants remaining.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999939.
77998 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.bed +
SampleData1/Fold_0/test_data.clumped.pruned.bim +
SampleData1/Fold_0/test_data.clumped.pruned.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
Pruned 402 variants from chromosome 1, leaving 5845.
Pruned 358 variants from chromosome 2, leaving 5637.
Pruned 333 variants from chromosome 3, leaving 4921.
Pruned 317 variants from chromosome 4, leaving 4649.
Pruned 329 variants from chromosome 5, leaving 4487.
Pruned 270 variants from chromosome 6, leaving 4147.
Pruned 284 variants from chromosome 7, leaving 3932.
Pruned 255 variants from chromosome 8, leaving 3613.
Pruned 208 variants from chromosome 9, leaving 3348.
Pruned 229 variants from chromosome 10, leaving 3723.
Pruned 236 variants from chromosome 11, leaving 3523.
Pruned 247 variants from chromosome 12, leaving 3469.
81%
Warning: At least 5268 duplicate IDs in --extract file.
82%
84%
85%
87%
88%
90%
91%
93%
95%
96%
98%
Pruned 170 variants from chromosome 13, leaving 2698.
1%
2%
4%
5%
7%
10%
11%
13%
15%
17%
18%
20%
22%
24%
26%
29%
31%
33%
35%
37%
39%
41%
42%
45%
47%
50%
52%
54%
56%
59%
61%
63%
65%
67%
69%
71%
73%
75%
76%
78%
80%
82%
84%
85%
87%
88%
90%
92%
94%
96%
98%
Pruned 124 variants from chromosome 14, leaving 2467.
1%
3%
4%
6%
7%
9%
11%
13%
14%
16%
17%
19%
21%
23%
25%
27%
30%
32%
34%
36%
38%
40%
42%
44%
46%
49%
51%
53%
55%
57%
60%
63%
65%
67%
69%
72%
74%
76%
79%
80%
82%
84%
85%
87%
88%
89%
91%
92%
94%
96%
98%
99%
Pruned 137 variants from chromosome 15, leaving 2469.
2%
4%
6%
8%
9%
11%
12%
14%
16%
17%
19%
21%
22%
24%
26%
27%
29%
31%
32%
34%
35%
36%
38%
40%
43%
44%
46%
47%
49%
50%
52%
53%
55%
58%
59%
61%
63%
66%
68%
69%
72%
74%
76%
77%
78%
80%
82%
84%
86%
88%
89%
90%
92%
93%
94%
96%
98%
Pruned 170 variants from chromosome 16, leaving 2702.
1%
3%
4%
6%
8%
10%
11%
14%
16%
17%
19%
20%
22%
23%
25%
26%
28%
31%
33%
35%
37%
39%
40%
42%
44%
46%
50%
52%
55%
56%
59%
60%
62%
64%
67%
69%
71%
73%
75%
77%
78%
80%
81%
83%
84%
86%
88%
89%
92%
93%
95%
97%
99%
Pruned 124 variants from chromosome 17, leaving 2544.
1%
3%
5%
6%
8%
9%
11%
13%
15%
16%
18%
19%
21%
23%
25%
27%
30%
31%
33%
35%
37%
40%
42%
43%
46%
48%
51%
53%
54%
56%
58%
60%
63%
64%
67%
68%
71%
73%
75%
77%
78%
80%
83%
85%
86%
88%
90%
92%
93%
95%
97%
99%
Pruned 134 variants from chromosome 18, leaving 2469.
2%
4%
6%
8%
10%
11%
14%
16%
18%
19%
23%
25%
28%
30%
33%
35%
37%
39%
43%
46%
48%
50%
52%
54%
56%
60%
63%
65%
69%
71%
74%
76%
79%
80%
83%
85%
87%
88%
90%
92%
95%
97%
Pruned 98 variants from chromosome 19, leaving 2038.
1%
3%
5%
7%
9%
11%
12%
14%
16%
18%
20%
22%
24%
26%
29%
31%
34%
36%
38%
41%
44%
48%
52%
54%
57%
58%
61%
64%
66%
68%
70%
73%
75%
76%
79%
81%
83%
85%
87%
89%
91%
93%
95%
97%
Pruned 147 variants from chromosome 20, leaving 2093.
3%
6%
10%
13%
17%
21%
23%
27%
31%
35%
38%
42%
46%
50%
53%
57%
61%
66%
69%
73%
76%
79%
83%
88%
91%
96%
Pruned 68 variants from chromosome 21, leaving 1263.
2%
5%
9%
12%
16%
21%
25%
28%
31%
36%
41%
45%
49%
52%
56%
60%
66%
70%
73%
76%
81%
84%
88%
90%
93%
97%
Pruned 57 variants from chromosome 22, leaving 1264.
Pruning complete. 4697 of 77998 variants removed.
Writing...
Marker lists written to SampleData1/Fold_0/test_data.clumped.pruned.prune.in
and SampleData1/Fold_0/test_data.clumped.pruned.prune.out .
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--extract SampleData1/Fold_0/train_data.valid.snp
--out SampleData1/Fold_0/test_data
--pca 6
63761 MB RAM detected; reserving 31880 MB for main workspace.
77998 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
--extract: 77998 variants remaining.
Using up to 8 threads (change this with --threads).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99% done.
Total genotyping rate is 0.999939.
77998 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
60 markers complete.
120 markers complete.
180 markers complete.
240 markers complete.
300 markers complete.
360 markers complete.
420 markers complete.
480 markers complete.
540 markers complete.
600 markers complete.
660 markers complete.
720 markers complete.
780 markers complete.
840 markers complete.
900 markers complete.
960 markers complete.
1020 markers complete.
1080 markers complete.
1140 markers complete.
1200 markers complete.
1260 markers complete.
1320 markers complete.
1380 markers complete.
1440 markers complete.
1500 markers complete.
1560 markers complete.
1620 markers complete.
1680 markers complete.
1740 markers complete.
1800 markers complete.
1860 markers complete.
1920 markers complete.
1980 markers complete.
2040 markers complete.
2100 markers complete.
2160 markers complete.
2220 markers complete.
2280 markers complete.
2340 markers complete.
2400 markers complete.
2460 markers complete.
2520 markers complete.
2580 markers complete.
2640 markers complete.
2700 markers complete.
2760 markers complete.
2820 markers complete.
2880 markers complete.
2940 markers complete.
3000 markers complete.
3060 markers complete.
3120 markers complete.
3180 markers complete.
3240 markers complete.
3300 markers complete.
3360 markers complete.
3420 markers complete.
3480 markers complete.
3540 markers complete.
3600 markers complete.
3660 markers complete.
3720 markers complete.
3780 markers complete.
3840 markers complete.
3900 markers complete.
3960 markers complete.
4020 markers complete.
4080 markers complete.
4140 markers complete.
4200 markers complete.
4260 markers complete.
4320 markers complete.
4380 markers complete.
4440 markers complete.
4500 markers complete.
4560 markers complete.
4620 markers complete.
4680 markers complete.
4740 markers complete.
4800 markers complete.
4860 markers complete.
4920 markers complete.
4980 markers complete.
5040 markers complete.
5100 markers complete.
5160 markers complete.
5220 markers complete.
5280 markers complete.
5340 markers complete.
5400 markers complete.
5460 markers complete.
5520 markers complete.
5580 markers complete.
5640 markers complete.
5700 markers complete.
5760 markers complete.
5820 markers complete.
5880 markers complete.
5940 markers complete.
6000 markers complete.
6060 markers complete.
6120 markers complete.
6180 markers complete.
6240 markers complete.
6300 markers complete.
6360 markers complete.
6420 markers complete.
6480 markers complete.
6540 markers complete.
6600 markers complete.
6660 markers complete.
6720 markers complete.
6780 markers complete.
6840 markers complete.
6900 markers complete.
6960 markers complete.
7020 markers complete.
7080 markers complete.
7140 markers complete.
7200 markers complete.
7260 markers complete.
7320 markers complete.
7380 markers complete.
7440 markers complete.
7500 markers complete.
7560 markers complete.
7620 markers complete.
7680 markers complete.
7740 markers complete.
7800 markers complete.
7860 markers complete.
7920 markers complete.
7980 markers complete.
8040 markers complete.
8100 markers complete.
8160 markers complete.
8220 markers complete.
8280 markers complete.
8340 markers complete.
8400 markers complete.
8460 markers complete.
8520 markers complete.
8580 markers complete.
8640 markers complete.
8700 markers complete.
8760 markers complete.
8820 markers complete.
8880 markers complete.
8940 markers complete.
9000 markers complete.
9060 markers complete.
9120 markers complete.
9180 markers complete.
9240 markers complete.
9300 markers complete.
9360 markers complete.
9420 markers complete.
9480 markers complete.
9540 markers complete.
9600 markers complete.
9660 markers complete.
9720 markers complete.
9780 markers complete.
9840 markers complete.
9900 markers complete.
9960 markers complete.
10020 markers complete.
10080 markers complete.
10140 markers complete.
10200 markers complete.
10260 markers complete.
10320 markers complete.
10380 markers complete.
10440 markers complete.
10500 markers complete.
10560 markers complete.
10620 markers complete.
10680 markers complete.
10740 markers complete.
10800 markers complete.
10860 markers complete.
10920 markers complete.
10980 markers complete.
11040 markers complete.
11100 markers complete.
11160 markers complete.
11220 markers complete.
11280 markers complete.
11340 markers complete.
11400 markers complete.
11460 markers complete.
11520 markers complete.
11580 markers complete.
11640 markers complete.
11700 markers complete.
11760 markers complete.
11820 markers complete.
11880 markers complete.
11940 markers complete.
12000 markers complete.
12060 markers complete.
12120 markers complete.
12180 markers complete.
12240 markers complete.
12300 markers complete.
12360 markers complete.
12420 markers complete.
12480 markers complete.
12540 markers complete.
12600 markers complete.
12660 markers complete.
12720 markers complete.
12780 markers complete.
12840 markers complete.
12900 markers complete.
12960 markers complete.
13020 markers complete.
13080 markers complete.
13140 markers complete.
13200 markers complete.
13260 markers complete.
13320 markers complete.
13380 markers complete.
13440 markers complete.
13500 markers complete.
13560 markers complete.
13620 markers complete.
13680 markers complete.
13740 markers complete.
13800 markers complete.
13860 markers complete.
13920 markers complete.
13980 markers complete.
14040 markers complete.
14100 markers complete.
14160 markers complete.
14220 markers complete.
14280 markers complete.
14340 markers complete.
14400 markers complete.
14460 markers complete.
14520 markers complete.
14580 markers complete.
14640 markers complete.
14700 markers complete.
14760 markers complete.
14820 markers complete.
14880 markers complete.
14940 markers complete.
15000 markers complete.
15060 markers complete.
15120 markers complete.
15180 markers complete.
15240 markers complete.
15300 markers complete.
15360 markers complete.
15420 markers complete.
15480 markers complete.
15540 markers complete.
15600 markers complete.
15660 markers complete.
15720 markers complete.
15780 markers complete.
15840 markers complete.
15900 markers complete.
15960 markers complete.
16020 markers complete.
16080 markers complete.
16140 markers complete.
16200 markers complete.
16260 markers complete.
16320 markers complete.
16380 markers complete.
16440 markers complete.
16500 markers complete.
16560 markers complete.
16620 markers complete.
16680 markers complete.
16740 markers complete.
16800 markers complete.
16860 markers complete.
16920 markers complete.
16980 markers complete.
17040 markers complete.
17100 markers complete.
17160 markers complete.
17220 markers complete.
17280 markers complete.
17340 markers complete.
17400 markers complete.
17460 markers complete.
17520 markers complete.
17580 markers complete.
17640 markers complete.
17700 markers complete.
17760 markers complete.
17820 markers complete.
17880 markers complete.
17940 markers complete.
18000 markers complete.
18060 markers complete.
18120 markers complete.
18180 markers complete.
18240 markers complete.
18300 markers complete.
18360 markers complete.
18420 markers complete.
18480 markers complete.
18540 markers complete.
18600 markers complete.
18660 markers complete.
18720 markers complete.
18780 markers complete.
18840 markers complete.
18900 markers complete.
18960 markers complete.
19020 markers complete.
19080 markers complete.
19140 markers complete.
19200 markers complete.
19260 markers complete.
19320 markers complete.
19380 markers complete.
19440 markers complete.
19500 markers complete.
19560 markers complete.
19620 markers complete.
19680 markers complete.
19740 markers complete.
19800 markers complete.
19860 markers complete.
19920 markers complete.
19980 markers complete.
20040 markers complete.
20100 markers complete.
20160 markers complete.
20220 markers complete.
20280 markers complete.
20340 markers complete.
20400 markers complete.
20460 markers complete.
20520 markers complete.
20580 markers complete.
20640 markers complete.
20700 markers complete.
20760 markers complete.
20820 markers complete.
20880 markers complete.
20940 markers complete.
21000 markers complete.
21060 markers complete.
21120 markers complete.
21180 markers complete.
21240 markers complete.
21300 markers complete.
21360 markers complete.
21420 markers complete.
21480 markers complete.
21540 markers complete.
21600 markers complete.
21660 markers complete.
21720 markers complete.
21780 markers complete.
21840 markers complete.
21900 markers complete.
21960 markers complete.
22020 markers complete.
22080 markers complete.
22140 markers complete.
22200 markers complete.
22260 markers complete.
22320 markers complete.
22380 markers complete.
22440 markers complete.
22500 markers complete.
22560 markers complete.
22620 markers complete.
22680 markers complete.
22740 markers complete.
22800 markers complete.
22860 markers complete.
22920 markers complete.
22980 markers complete.
23040 markers complete.
23100 markers complete.
23160 markers complete.
23220 markers complete.
23280 markers complete.
23340 markers complete.
23400 markers complete.
23460 markers complete.
23520 markers complete.
23580 markers complete.
23640 markers complete.
23700 markers complete.
23760 markers complete.
23820 markers complete.
23880 markers complete.
23940 markers complete.
24000 markers complete.
24060 markers complete.
24120 markers complete.
24180 markers complete.
24240 markers complete.
24300 markers complete.
24360 markers complete.
24420 markers complete.
24480 markers complete.
24540 markers complete.
24600 markers complete.
24660 markers complete.
24720 markers complete.
24780 markers complete.
24840 markers complete.
24900 markers complete.
24960 markers complete.
25020 markers complete.
25080 markers complete.
25140 markers complete.
25200 markers complete.
25260 markers complete.
25320 markers complete.
25380 markers complete.
25440 markers complete.
25500 markers complete.
25560 markers complete.
25620 markers complete.
25680 markers complete.
25740 markers complete.
25800 markers complete.
25860 markers complete.
25920 markers complete.
25980 markers complete.
26040 markers complete.
26100 markers complete.
26160 markers complete.
26220 markers complete.
26280 markers complete.
26340 markers complete.
26400 markers complete.
26460 markers complete.
26520 markers complete.
26580 markers complete.
26640 markers complete.
26700 markers complete.
26760 markers complete.
26820 markers complete.
26880 markers complete.
26940 markers complete.
27000 markers complete.
27060 markers complete.
27120 markers complete.
27180 markers complete.
27240 markers complete.
27300 markers complete.
27360 markers complete.
27420 markers complete.
27480 markers complete.
27540 markers complete.
27600 markers complete.
27660 markers complete.
27720 markers complete.
27780 markers complete.
27840 markers complete.
27900 markers complete.
27960 markers complete.
28020 markers complete.
28080 markers complete.
28140 markers complete.
28200 markers complete.
28260 markers complete.
28320 markers complete.
28380 markers complete.
28440 markers complete.
28500 markers complete.
28560 markers complete.
28620 markers complete.
28680 markers complete.
28740 markers complete.
28800 markers complete.
28860 markers complete.
28920 markers complete.
28980 markers complete.
29040 markers complete.
29100 markers complete.
29160 markers complete.
29220 markers complete.
29280 markers complete.
29340 markers complete.
29400 markers complete.
29460 markers complete.
29520 markers complete.
29580 markers complete.
29640 markers complete.
29700 markers complete.
29760 markers complete.
29820 markers complete.
29880 markers complete.
29940 markers complete.
30000 markers complete.
30060 markers complete.
30120 markers complete.
30180 markers complete.
30240 markers complete.
30300 markers complete.
30360 markers complete.
30420 markers complete.
30480 markers complete.
30540 markers complete.
30600 markers complete.
30660 markers complete.
30720 markers complete.
30780 markers complete.
30840 markers complete.
30900 markers complete.
30960 markers complete.
31020 markers complete.
31080 markers complete.
31140 markers complete.
31200 markers complete.
31260 markers complete.
31320 markers complete.
31380 markers complete.
31440 markers complete.
31500 markers complete.
31560 markers complete.
31620 markers complete.
31680 markers complete.
31740 markers complete.
31800 markers complete.
31860 markers complete.
31920 markers complete.
31980 markers complete.
32040 markers complete.
32100 markers complete.
32160 markers complete.
32220 markers complete.
32280 markers complete.
32340 markers complete.
32400 markers complete.
32460 markers complete.
32520 markers complete.
32580 markers complete.
32640 markers complete.
32700 markers complete.
32760 markers complete.
32820 markers complete.
32880 markers complete.
32940 markers complete.
33000 markers complete.
33060 markers complete.
33120 markers complete.
33180 markers complete.
33240 markers complete.
33300 markers complete.
33360 markers complete.
33420 markers complete.
33480 markers complete.
33540 markers complete.
33600 markers complete.
33660 markers complete.
33720 markers complete.
33780 markers complete.
33840 markers complete.
33900 markers complete.
33960 markers complete.
34020 markers complete.
34080 markers complete.
34140 markers complete.
34200 markers complete.
34260 markers complete.
34320 markers complete.
34380 markers complete.
34440 markers complete.
34500 markers complete.
34560 markers complete.
34620 markers complete.
34680 markers complete.
34740 markers complete.
34800 markers complete.
34860 markers complete.
34920 markers complete.
34980 markers complete.
35040 markers complete.
35100 markers complete.
35160 markers complete.
35220 markers complete.
35280 markers complete.
35340 markers complete.
35400 markers complete.
35460 markers complete.
35520 markers complete.
35580 markers complete.
35640 markers complete.
35700 markers complete.
35760 markers complete.
35820 markers complete.
35880 markers complete.
35940 markers complete.
36000 markers complete.
36060 markers complete.
36120 markers complete.
36180 markers complete.
36240 markers complete.
36300 markers complete.
36360 markers complete.
36420 markers complete.
36480 markers complete.
36540 markers complete.
36600 markers complete.
36660 markers complete.
36720 markers complete.
36780 markers complete.
36840 markers complete.
36900 markers complete.
36960 markers complete.
37020 markers complete.
37080 markers complete.
37140 markers complete.
37200 markers complete.
37260 markers complete.
37320 markers complete.
37380 markers complete.
37440 markers complete.
37500 markers complete.
37560 markers complete.
37620 markers complete.
37680 markers complete.
37740 markers complete.
37800 markers complete.
37860 markers complete.
37920 markers complete.
37980 markers complete.
38040 markers complete.
38100 markers complete.
38160 markers complete.
38220 markers complete.
38280 markers complete.
38340 markers complete.
38400 markers complete.
38460 markers complete.
38520 markers complete.
38580 markers complete.
38640 markers complete.
38700 markers complete.
38760 markers complete.
38820 markers complete.
38880 markers complete.
38940 markers complete.
39000 markers complete.
39060 markers complete.
39120 markers complete.
39180 markers complete.
39240 markers complete.
39300 markers complete.
39360 markers complete.
39420 markers complete.
39480 markers complete.
39540 markers complete.
39600 markers complete.
39660 markers complete.
39720 markers complete.
39780 markers complete.
39840 markers complete.
39900 markers complete.
39960 markers complete.
40020 markers complete.
40080 markers complete.
40140 markers complete.
40200 markers complete.
40260 markers complete.
40320 markers complete.
40380 markers complete.
40440 markers complete.
40500 markers complete.
40560 markers complete.
40620 markers complete.
40680 markers complete.
40740 markers complete.
40800 markers complete.
40860 markers complete.
40920 markers complete.
40980 markers complete.
41040 markers complete.
41100 markers complete.
41160 markers complete.
41220 markers complete.
41280 markers complete.
41340 markers complete.
41400 markers complete.
41460 markers complete.
41520 markers complete.
41580 markers complete.
41640 markers complete.
41700 markers complete.
41760 markers complete.
41820 markers complete.
41880 markers complete.
41940 markers complete.
42000 markers complete.
42060 markers complete.
42120 markers complete.
42180 markers complete.
42240 markers complete.
42300 markers complete.
42360 markers complete.
42420 markers complete.
42480 markers complete.
42540 markers complete.
42600 markers complete.
42660 markers complete.
42720 markers complete.
42780 markers complete.
42840 markers complete.
42900 markers complete.
42960 markers complete.
43020 markers complete.
43080 markers complete.
43140 markers complete.
43200 markers complete.
43260 markers complete.
43320 markers complete.
43380 markers complete.
43440 markers complete.
43500 markers complete.
43560 markers complete.
43620 markers complete.
43680 markers complete.
43740 markers complete.
43800 markers complete.
43860 markers complete.
43920 markers complete.
43980 markers complete.
44040 markers complete.
44100 markers complete.
44160 markers complete.
44220 markers complete.
44280 markers complete.
44340 markers complete.
44400 markers complete.
44460 markers complete.
44520 markers complete.
44580 markers complete.
44640 markers complete.
44700 markers complete.
44760 markers complete.
44820 markers complete.
44880 markers complete.
44940 markers complete.
45000 markers complete.
45060 markers complete.
45120 markers complete.
45180 markers complete.
45240 markers complete.
45300 markers complete.
45360 markers complete.
45420 markers complete.
45480 markers complete.
45540 markers complete.
45600 markers complete.
45660 markers complete.
45720 markers complete.
45780 markers complete.
45840 markers complete.
45900 markers complete.
45960 markers complete.
46020 markers complete.
46080 markers complete.
46140 markers complete.
46200 markers complete.
46260 markers complete.
46320 markers complete.
46380 markers complete.
46440 markers complete.
46500 markers complete.
46560 markers complete.
46620 markers complete.
46680 markers complete.
46740 markers complete.
46800 markers complete.
46860 markers complete.
46920 markers complete.
46980 markers complete.
47040 markers complete.
47100 markers complete.
47160 markers complete.
47220 markers complete.
47280 markers complete.
47340 markers complete.
47400 markers complete.
47460 markers complete.
47520 markers complete.
47580 markers complete.
47640 markers complete.
47700 markers complete.
47760 markers complete.
47820 markers complete.
47880 markers complete.
47940 markers complete.
48000 markers complete.
48060 markers complete.
48120 markers complete.
48180 markers complete.
48240 markers complete.
48300 markers complete.
48360 markers complete.
48420 markers complete.
48480 markers complete.
48540 markers complete.
48600 markers complete.
48660 markers complete.
48720 markers complete.
48780 markers complete.
48840 markers complete.
48900 markers complete.
48960 markers complete.
49020 markers complete.
49080 markers complete.
49140 markers complete.
49200 markers complete.
49260 markers complete.
49320 markers complete.
49380 markers complete.
49440 markers complete.
49500 markers complete.
49560 markers complete.
49620 markers complete.
49680 markers complete.
49740 markers complete.
49800 markers complete.
49860 markers complete.
49920 markers complete.
49980 markers complete.
50040 markers complete.
50100 markers complete.
50160 markers complete.
50220 markers complete.
50280 markers complete.
50340 markers complete.
50400 markers complete.
50460 markers complete.
50520 markers complete.
50580 markers complete.
50640 markers complete.
50700 markers complete.
50760 markers complete.
50820 markers complete.
50880 markers complete.
50940 markers complete.
51000 markers complete.
51060 markers complete.
51120 markers complete.
51180 markers complete.
51240 markers complete.
51300 markers complete.
51360 markers complete.
51420 markers complete.
51480 markers complete.
51540 markers complete.
51600 markers complete.
51660 markers complete.
51720 markers complete.
51780 markers complete.
51840 markers complete.
51900 markers complete.
51960 markers complete.
52020 markers complete.
52080 markers complete.
52140 markers complete.
52200 markers complete.
52260 markers complete.
52320 markers complete.
52380 markers complete.
52440 markers complete.
52500 markers complete.
52560 markers complete.
52620 markers complete.
52680 markers complete.
52740 markers complete.
52800 markers complete.
52860 markers complete.
52920 markers complete.
52980 markers complete.
53040 markers complete.
53100 markers complete.
53160 markers complete.
53220 markers complete.
53280 markers complete.
53340 markers complete.
53400 markers complete.
53460 markers complete.
53520 markers complete.
53580 markers complete.
53640 markers complete.
53700 markers complete.
Warning: At least 5268 duplicate IDs in --extract file.
53760 markers complete.
53820 markers complete.
53880 markers complete.
53940 markers complete.
54000 markers complete.
54060 markers complete.
54120 markers complete.
54180 markers complete.
54240 markers complete.
54300 markers complete.
54360 markers complete.
54420 markers complete.
54480 markers complete.
54540 markers complete.
54600 markers complete.
54660 markers complete.
54720 markers complete.
54780 markers complete.
54840 markers complete.
54900 markers complete.
54960 markers complete.
55020 markers complete.
55080 markers complete.
55140 markers complete.
55200 markers complete.
55260 markers complete.
55320 markers complete.
55380 markers complete.
55440 markers complete.
55500 markers complete.
55560 markers complete.
55620 markers complete.
55680 markers complete.
55740 markers complete.
55800 markers complete.
55860 markers complete.
55920 markers complete.
55980 markers complete.
56040 markers complete.
56100 markers complete.
56160 markers complete.
56220 markers complete.
56280 markers complete.
56340 markers complete.
56400 markers complete.
56460 markers complete.
56520 markers complete.
56580 markers complete.
56640 markers complete.
56700 markers complete.
56760 markers complete.
56820 markers complete.
56880 markers complete.
56940 markers complete.
57000 markers complete.
57060 markers complete.
57120 markers complete.
57180 markers complete.
57240 markers complete.
57300 markers complete.
57360 markers complete.
57420 markers complete.
57480 markers complete.
57540 markers complete.
57600 markers complete.
57660 markers complete.
57720 markers complete.
57780 markers complete.
57840 markers complete.
57900 markers complete.
57960 markers complete.
58020 markers complete.
58080 markers complete.
58140 markers complete.
58200 markers complete.
58260 markers complete.
58320 markers complete.
58380 markers complete.
58440 markers complete.
58500 markers complete.
58560 markers complete.
58620 markers complete.
58680 markers complete.
58740 markers complete.
58800 markers complete.
58860 markers complete.
58920 markers complete.
58980 markers complete.
59040 markers complete.
59100 markers complete.
59160 markers complete.
59220 markers complete.
59280 markers complete.
59340 markers complete.
59400 markers complete.
59460 markers complete.
59520 markers complete.
59580 markers complete.
59640 markers complete.
59700 markers complete.
59760 markers complete.
59820 markers complete.
59880 markers complete.
59940 markers complete.
60000 markers complete.
60060 markers complete.
60120 markers complete.
60180 markers complete.
60240 markers complete.
60300 markers complete.
60360 markers complete.
60420 markers complete.
60480 markers complete.
60540 markers complete.
60600 markers complete.
60660 markers complete.
60720 markers complete.
60780 markers complete.
60840 markers complete.
60900 markers complete.
60960 markers complete.
61020 markers complete.
61080 markers complete.
61140 markers complete.
61200 markers complete.
61260 markers complete.
61320 markers complete.
61380 markers complete.
61440 markers complete.
61500 markers complete.
61560 markers complete.
61620 markers complete.
61680 markers complete.
61740 markers complete.
61800 markers complete.
61860 markers complete.
61920 markers complete.
61980 markers complete.
62040 markers complete.
62100 markers complete.
62160 markers complete.
62220 markers complete.
62280 markers complete.
62340 markers complete.
62400 markers complete.
62460 markers complete.
62520 markers complete.
62580 markers complete.
62640 markers complete.
62700 markers complete.
62760 markers complete.
62820 markers complete.
62880 markers complete.
62940 markers complete.
63000 markers complete.
63060 markers complete.
63120 markers complete.
63180 markers complete.
63240 markers complete.
63300 markers complete.
63360 markers complete.
63420 markers complete.
63480 markers complete.
63540 markers complete.
63600 markers complete.
63660 markers complete.
63720 markers complete.
63780 markers complete.
63840 markers complete.
63900 markers complete.
63960 markers complete.
64020 markers complete.
64080 markers complete.
64140 markers complete.
64200 markers complete.
64260 markers complete.
64320 markers complete.
64380 markers complete.
64440 markers complete.
64500 markers complete.
64560 markers complete.
64620 markers complete.
64680 markers complete.
64740 markers complete.
64800 markers complete.
64860 markers complete.
64920 markers complete.
64980 markers complete.
65040 markers complete.
65100 markers complete.
65160 markers complete.
65220 markers complete.
65280 markers complete.
65340 markers complete.
65400 markers complete.
65460 markers complete.
65520 markers complete.
65580 markers complete.
65640 markers complete.
65700 markers complete.
65760 markers complete.
65820 markers complete.
65880 markers complete.
65940 markers complete.
66000 markers complete.
66060 markers complete.
66120 markers complete.
66180 markers complete.
66240 markers complete.
66300 markers complete.
66360 markers complete.
66420 markers complete.
66480 markers complete.
66540 markers complete.
66600 markers complete.
66660 markers complete.
66720 markers complete.
66780 markers complete.
66840 markers complete.
66900 markers complete.
66960 markers complete.
67020 markers complete.
67080 markers complete.
67140 markers complete.
67200 markers complete.
67260 markers complete.
67320 markers complete.
67380 markers complete.
67440 markers complete.
67500 markers complete.
67560 markers complete.
67620 markers complete.
67680 markers complete.
67740 markers complete.
67800 markers complete.
67860 markers complete.
67920 markers complete.
67980 markers complete.
68040 markers complete.
68100 markers complete.
68160 markers complete.
68220 markers complete.
68280 markers complete.
68340 markers complete.
68400 markers complete.
68460 markers complete.
68520 markers complete.
68580 markers complete.
68640 markers complete.
68700 markers complete.
68760 markers complete.
68820 markers complete.
68880 markers complete.
68940 markers complete.
69000 markers complete.
69060 markers complete.
69120 markers complete.
69180 markers complete.
69240 markers complete.
69300 markers complete.
69360 markers complete.
69420 markers complete.
69480 markers complete.
69540 markers complete.
69600 markers complete.
69660 markers complete.
69720 markers complete.
69780 markers complete.
69840 markers complete.
69900 markers complete.
69960 markers complete.
70020 markers complete.
70080 markers complete.
70140 markers complete.
70200 markers complete.
70260 markers complete.
70320 markers complete.
70380 markers complete.
70440 markers complete.
70500 markers complete.
70560 markers complete.
70620 markers complete.
70680 markers complete.
70740 markers complete.
70800 markers complete.
70860 markers complete.
70920 markers complete.
70980 markers complete.
71040 markers complete.
71100 markers complete.
71160 markers complete.
71220 markers complete.
71280 markers complete.
71340 markers complete.
71400 markers complete.
71460 markers complete.
71520 markers complete.
71580 markers complete.
71640 markers complete.
71700 markers complete.
71760 markers complete.
71820 markers complete.
71880 markers complete.
71940 markers complete.
72000 markers complete.
72060 markers complete.
72120 markers complete.
72180 markers complete.
72240 markers complete.
72300 markers complete.
72360 markers complete.
72420 markers complete.
72480 markers complete.
72540 markers complete.
72600 markers complete.
72660 markers complete.
72720 markers complete.
72780 markers complete.
72840 markers complete.
72900 markers complete.
72960 markers complete.
73020 markers complete.
73080 markers complete.
73140 markers complete.
73200 markers complete.
73260 markers complete.
73320 markers complete.
73380 markers complete.
73440 markers complete.
73500 markers complete.
73560 markers complete.
73620 markers complete.
73680 markers complete.
73740 markers complete.
73800 markers complete.
73860 markers complete.
73920 markers complete.
73980 markers complete.
74040 markers complete.
74100 markers complete.
74160 markers complete.
74220 markers complete.
74280 markers complete.
74340 markers complete.
74400 markers complete.
74460 markers complete.
74520 markers complete.
74580 markers complete.
74640 markers complete.
74700 markers complete.
74760 markers complete.
74820 markers complete.
74880 markers complete.
74940 markers complete.
75000 markers complete.
75060 markers complete.
75120 markers complete.
75180 markers complete.
75240 markers complete.
75300 markers complete.
75360 markers complete.
75420 markers complete.
75480 markers complete.
75540 markers complete.
75600 markers complete.
75660 markers complete.
75720 markers complete.
75780 markers complete.
75840 markers complete.
75900 markers complete.
75960 markers complete.
76020 markers complete.
76080 markers complete.
76140 markers complete.
76200 markers complete.
76260 markers complete.
76320 markers complete.
76380 markers complete.
76440 markers complete.
76500 markers complete.
76560 markers complete.
76620 markers complete.
76680 markers complete.
76740 markers complete.
76800 markers complete.
76860 markers complete.
76920 markers complete.
76980 markers complete.
77040 markers complete.
77100 markers complete.
77160 markers complete.
77220 markers complete.
77280 markers complete.
77340 markers complete.
77400 markers complete.
77460 markers complete.
77520 markers complete.
77580 markers complete.
77640 markers complete.
77700 markers complete.
77760 markers complete.
77820 markers complete.
77880 markers complete.
77940 markers complete.
77998 markers complete.
Relationship matrix calculation complete.
[extracting eigenvalues and eigenvectors]
--pca: Results saved to SampleData1/Fold_0/test_data.eigenval and
SampleData1/Fold_0/test_data.eigenvec .
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--extract SampleData1/Fold_0/train_data.valid.snp
--out SampleData1/Fold_0/train_data
--pca 6
63761 MB RAM detected; reserving 31880 MB for main workspace.
77998 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
--extract: 77998 variants remaining.
Using up to 8 threads (change this with --threads).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99% done.
Total genotyping rate is 0.999939.
77998 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
60 markers complete.
120 markers complete.
180 markers complete.
240 markers complete.
300 markers complete.
360 markers complete.
420 markers complete.
480 markers complete.
540 markers complete.
600 markers complete.
660 markers complete.
720 markers complete.
780 markers complete.
840 markers complete.
900 markers complete.
960 markers complete.
1020 markers complete.
1080 markers complete.
1140 markers complete.
1200 markers complete.
1260 markers complete.
1320 markers complete.
1380 markers complete.
1440 markers complete.
1500 markers complete.
1560 markers complete.
1620 markers complete.
1680 markers complete.
1740 markers complete.
1800 markers complete.
1860 markers complete.
1920 markers complete.
1980 markers complete.
2040 markers complete.
2100 markers complete.
2160 markers complete.
2220 markers complete.
2280 markers complete.
2340 markers complete.
2400 markers complete.
2460 markers complete.
2520 markers complete.
2580 markers complete.
2640 markers complete.
2700 markers complete.
2760 markers complete.
2820 markers complete.
2880 markers complete.
2940 markers complete.
3000 markers complete.
3060 markers complete.
3120 markers complete.
3180 markers complete.
3240 markers complete.
3300 markers complete.
3360 markers complete.
3420 markers complete.
3480 markers complete.
3540 markers complete.
3600 markers complete.
3660 markers complete.
3720 markers complete.
3780 markers complete.
3840 markers complete.
3900 markers complete.
3960 markers complete.
4020 markers complete.
4080 markers complete.
4140 markers complete.
4200 markers complete.
4260 markers complete.
4320 markers complete.
4380 markers complete.
4440 markers complete.
4500 markers complete.
4560 markers complete.
4620 markers complete.
4680 markers complete.
4740 markers complete.
4800 markers complete.
4860 markers complete.
4920 markers complete.
4980 markers complete.
5040 markers complete.
5100 markers complete.
5160 markers complete.
5220 markers complete.
5280 markers complete.
5340 markers complete.
5400 markers complete.
5460 markers complete.
5520 markers complete.
5580 markers complete.
5640 markers complete.
5700 markers complete.
Warning: At least 5268 duplicate IDs in --extract file.
5760 markers complete.
5820 markers complete.
5880 markers complete.
5940 markers complete.
6000 markers complete.
6060 markers complete.
6120 markers complete.
6180 markers complete.
6240 markers complete.
6300 markers complete.
6360 markers complete.
6420 markers complete.
6480 markers complete.
6540 markers complete.
6600 markers complete.
6660 markers complete.
6720 markers complete.
6780 markers complete.
6840 markers complete.
6900 markers complete.
6960 markers complete.
7020 markers complete.
7080 markers complete.
7140 markers complete.
7200 markers complete.
7260 markers complete.
7320 markers complete.
7380 markers complete.
7440 markers complete.
7500 markers complete.
7560 markers complete.
7620 markers complete.
7680 markers complete.
7740 markers complete.
7800 markers complete.
7860 markers complete.
7920 markers complete.
7980 markers complete.
8040 markers complete.
8100 markers complete.
8160 markers complete.
8220 markers complete.
8280 markers complete.
8340 markers complete.
8400 markers complete.
8460 markers complete.
8520 markers complete.
8580 markers complete.
8640 markers complete.
8700 markers complete.
8760 markers complete.
8820 markers complete.
8880 markers complete.
8940 markers complete.
9000 markers complete.
9060 markers complete.
9120 markers complete.
9180 markers complete.
9240 markers complete.
9300 markers complete.
9360 markers complete.
9420 markers complete.
9480 markers complete.
9540 markers complete.
9600 markers complete.
9660 markers complete.
9720 markers complete.
9780 markers complete.
9840 markers complete.
9900 markers complete.
9960 markers complete.
10020 markers complete.
10080 markers complete.
10140 markers complete.
10200 markers complete.
10260 markers complete.
10320 markers complete.
10380 markers complete.
10440 markers complete.
10500 markers complete.
10560 markers complete.
10620 markers complete.
10680 markers complete.
10740 markers complete.
10800 markers complete.
10860 markers complete.
10920 markers complete.
10980 markers complete.
11040 markers complete.
11100 markers complete.
11160 markers complete.
11220 markers complete.
11280 markers complete.
11340 markers complete.
11400 markers complete.
11460 markers complete.
11520 markers complete.
11580 markers complete.
11640 markers complete.
11700 markers complete.
11760 markers complete.
11820 markers complete.
11880 markers complete.
11940 markers complete.
12000 markers complete.
12060 markers complete.
12120 markers complete.
12180 markers complete.
12240 markers complete.
12300 markers complete.
12360 markers complete.
12420 markers complete.
12480 markers complete.
12540 markers complete.
12600 markers complete.
12660 markers complete.
12720 markers complete.
12780 markers complete.
12840 markers complete.
12900 markers complete.
12960 markers complete.
13020 markers complete.
13080 markers complete.
13140 markers complete.
13200 markers complete.
13260 markers complete.
13320 markers complete.
13380 markers complete.
13440 markers complete.
13500 markers complete.
13560 markers complete.
13620 markers complete.
13680 markers complete.
13740 markers complete.
13800 markers complete.
13860 markers complete.
13920 markers complete.
13980 markers complete.
14040 markers complete.
14100 markers complete.
14160 markers complete.
14220 markers complete.
14280 markers complete.
14340 markers complete.
14400 markers complete.
14460 markers complete.
14520 markers complete.
14580 markers complete.
14640 markers complete.
14700 markers complete.
14760 markers complete.
14820 markers complete.
14880 markers complete.
14940 markers complete.
15000 markers complete.
15060 markers complete.
15120 markers complete.
15180 markers complete.
15240 markers complete.
15300 markers complete.
15360 markers complete.
15420 markers complete.
15480 markers complete.
15540 markers complete.
15600 markers complete.
15660 markers complete.
15720 markers complete.
15780 markers complete.
15840 markers complete.
15900 markers complete.
15960 markers complete.
16020 markers complete.
16080 markers complete.
16140 markers complete.
16200 markers complete.
16260 markers complete.
16320 markers complete.
16380 markers complete.
16440 markers complete.
16500 markers complete.
16560 markers complete.
16620 markers complete.
16680 markers complete.
16740 markers complete.
16800 markers complete.
16860 markers complete.
16920 markers complete.
16980 markers complete.
17040 markers complete.
17100 markers complete.
17160 markers complete.
17220 markers complete.
17280 markers complete.
17340 markers complete.
17400 markers complete.
17460 markers complete.
17520 markers complete.
17580 markers complete.
17640 markers complete.
17700 markers complete.
17760 markers complete.
17820 markers complete.
17880 markers complete.
17940 markers complete.
18000 markers complete.
18060 markers complete.
18120 markers complete.
18180 markers complete.
18240 markers complete.
18300 markers complete.
18360 markers complete.
18420 markers complete.
18480 markers complete.
18540 markers complete.
18600 markers complete.
18660 markers complete.
18720 markers complete.
18780 markers complete.
18840 markers complete.
18900 markers complete.
18960 markers complete.
19020 markers complete.
19080 markers complete.
19140 markers complete.
19200 markers complete.
19260 markers complete.
19320 markers complete.
19380 markers complete.
19440 markers complete.
19500 markers complete.
19560 markers complete.
19620 markers complete.
19680 markers complete.
19740 markers complete.
19800 markers complete.
19860 markers complete.
19920 markers complete.
19980 markers complete.
20040 markers complete.
20100 markers complete.
20160 markers complete.
20220 markers complete.
20280 markers complete.
20340 markers complete.
20400 markers complete.
20460 markers complete.
20520 markers complete.
20580 markers complete.
20640 markers complete.
20700 markers complete.
20760 markers complete.
20820 markers complete.
20880 markers complete.
20940 markers complete.
21000 markers complete.
21060 markers complete.
21120 markers complete.
21180 markers complete.
21240 markers complete.
21300 markers complete.
21360 markers complete.
21420 markers complete.
21480 markers complete.
21540 markers complete.
21600 markers complete.
21660 markers complete.
21720 markers complete.
21780 markers complete.
21840 markers complete.
21900 markers complete.
21960 markers complete.
22020 markers complete.
22080 markers complete.
22140 markers complete.
22200 markers complete.
22260 markers complete.
22320 markers complete.
22380 markers complete.
22440 markers complete.
22500 markers complete.
22560 markers complete.
22620 markers complete.
22680 markers complete.
22740 markers complete.
22800 markers complete.
22860 markers complete.
22920 markers complete.
22980 markers complete.
23040 markers complete.
23100 markers complete.
23160 markers complete.
23220 markers complete.
23280 markers complete.
23340 markers complete.
23400 markers complete.
23460 markers complete.
23520 markers complete.
23580 markers complete.
23640 markers complete.
23700 markers complete.
23760 markers complete.
23820 markers complete.
23880 markers complete.
23940 markers complete.
24000 markers complete.
24060 markers complete.
24120 markers complete.
24180 markers complete.
24240 markers complete.
24300 markers complete.
24360 markers complete.
24420 markers complete.
24480 markers complete.
24540 markers complete.
24600 markers complete.
24660 markers complete.
24720 markers complete.
24780 markers complete.
24840 markers complete.
24900 markers complete.
24960 markers complete.
25020 markers complete.
25080 markers complete.
25140 markers complete.
25200 markers complete.
25260 markers complete.
25320 markers complete.
25380 markers complete.
25440 markers complete.
25500 markers complete.
25560 markers complete.
25620 markers complete.
25680 markers complete.
25740 markers complete.
25800 markers complete.
25860 markers complete.
25920 markers complete.
25980 markers complete.
26040 markers complete.
26100 markers complete.
26160 markers complete.
26220 markers complete.
26280 markers complete.
26340 markers complete.
26400 markers complete.
26460 markers complete.
26520 markers complete.
26580 markers complete.
26640 markers complete.
26700 markers complete.
26760 markers complete.
26820 markers complete.
26880 markers complete.
26940 markers complete.
27000 markers complete.
27060 markers complete.
27120 markers complete.
27180 markers complete.
27240 markers complete.
27300 markers complete.
27360 markers complete.
27420 markers complete.
27480 markers complete.
27540 markers complete.
27600 markers complete.
27660 markers complete.
27720 markers complete.
27780 markers complete.
27840 markers complete.
27900 markers complete.
27960 markers complete.
28020 markers complete.
28080 markers complete.
28140 markers complete.
28200 markers complete.
28260 markers complete.
28320 markers complete.
28380 markers complete.
28440 markers complete.
28500 markers complete.
28560 markers complete.
28620 markers complete.
28680 markers complete.
28740 markers complete.
28800 markers complete.
28860 markers complete.
28920 markers complete.
28980 markers complete.
29040 markers complete.
29100 markers complete.
29160 markers complete.
29220 markers complete.
29280 markers complete.
29340 markers complete.
29400 markers complete.
29460 markers complete.
29520 markers complete.
29580 markers complete.
29640 markers complete.
29700 markers complete.
29760 markers complete.
29820 markers complete.
29880 markers complete.
29940 markers complete.
30000 markers complete.
30060 markers complete.
30120 markers complete.
30180 markers complete.
30240 markers complete.
30300 markers complete.
30360 markers complete.
30420 markers complete.
30480 markers complete.
30540 markers complete.
30600 markers complete.
30660 markers complete.
30720 markers complete.
30780 markers complete.
30840 markers complete.
30900 markers complete.
30960 markers complete.
31020 markers complete.
31080 markers complete.
31140 markers complete.
31200 markers complete.
31260 markers complete.
31320 markers complete.
31380 markers complete.
31440 markers complete.
31500 markers complete.
31560 markers complete.
31620 markers complete.
31680 markers complete.
31740 markers complete.
31800 markers complete.
31860 markers complete.
31920 markers complete.
31980 markers complete.
32040 markers complete.
32100 markers complete.
32160 markers complete.
32220 markers complete.
32280 markers complete.
32340 markers complete.
32400 markers complete.
32460 markers complete.
32520 markers complete.
32580 markers complete.
32640 markers complete.
32700 markers complete.
32760 markers complete.
32820 markers complete.
32880 markers complete.
32940 markers complete.
33000 markers complete.
33060 markers complete.
33120 markers complete.
33180 markers complete.
33240 markers complete.
33300 markers complete.
33360 markers complete.
33420 markers complete.
33480 markers complete.
33540 markers complete.
33600 markers complete.
33660 markers complete.
33720 markers complete.
33780 markers complete.
33840 markers complete.
33900 markers complete.
33960 markers complete.
34020 markers complete.
34080 markers complete.
34140 markers complete.
34200 markers complete.
34260 markers complete.
34320 markers complete.
34380 markers complete.
34440 markers complete.
34500 markers complete.
34560 markers complete.
34620 markers complete.
34680 markers complete.
34740 markers complete.
34800 markers complete.
34860 markers complete.
34920 markers complete.
34980 markers complete.
35040 markers complete.
35100 markers complete.
35160 markers complete.
35220 markers complete.
35280 markers complete.
35340 markers complete.
35400 markers complete.
35460 markers complete.
35520 markers complete.
35580 markers complete.
35640 markers complete.
35700 markers complete.
35760 markers complete.
35820 markers complete.
35880 markers complete.
35940 markers complete.
36000 markers complete.
36060 markers complete.
36120 markers complete.
36180 markers complete.
36240 markers complete.
36300 markers complete.
36360 markers complete.
36420 markers complete.
36480 markers complete.
36540 markers complete.
36600 markers complete.
36660 markers complete.
36720 markers complete.
36780 markers complete.
36840 markers complete.
36900 markers complete.
36960 markers complete.
37020 markers complete.
37080 markers complete.
37140 markers complete.
37200 markers complete.
37260 markers complete.
37320 markers complete.
37380 markers complete.
37440 markers complete.
37500 markers complete.
37560 markers complete.
37620 markers complete.
37680 markers complete.
37740 markers complete.
37800 markers complete.
37860 markers complete.
37920 markers complete.
37980 markers complete.
38040 markers complete.
38100 markers complete.
38160 markers complete.
38220 markers complete.
38280 markers complete.
38340 markers complete.
38400 markers complete.
38460 markers complete.
38520 markers complete.
38580 markers complete.
38640 markers complete.
38700 markers complete.
38760 markers complete.
38820 markers complete.
38880 markers complete.
38940 markers complete.
39000 markers complete.
39060 markers complete.
39120 markers complete.
39180 markers complete.
39240 markers complete.
39300 markers complete.
39360 markers complete.
39420 markers complete.
39480 markers complete.
39540 markers complete.
39600 markers complete.
39660 markers complete.
39720 markers complete.
39780 markers complete.
39840 markers complete.
39900 markers complete.
39960 markers complete.
40020 markers complete.
40080 markers complete.
40140 markers complete.
40200 markers complete.
40260 markers complete.
40320 markers complete.
40380 markers complete.
40440 markers complete.
40500 markers complete.
40560 markers complete.
40620 markers complete.
40680 markers complete.
40740 markers complete.
40800 markers complete.
40860 markers complete.
40920 markers complete.
40980 markers complete.
41040 markers complete.
41100 markers complete.
41160 markers complete.
41220 markers complete.
41280 markers complete.
41340 markers complete.
41400 markers complete.
41460 markers complete.
41520 markers complete.
41580 markers complete.
41640 markers complete.
41700 markers complete.
41760 markers complete.
41820 markers complete.
41880 markers complete.
41940 markers complete.
42000 markers complete.
42060 markers complete.
42120 markers complete.
42180 markers complete.
42240 markers complete.
42300 markers complete.
42360 markers complete.
42420 markers complete.
42480 markers complete.
42540 markers complete.
42600 markers complete.
42660 markers complete.
42720 markers complete.
42780 markers complete.
42840 markers complete.
42900 markers complete.
42960 markers complete.
43020 markers complete.
43080 markers complete.
43140 markers complete.
43200 markers complete.
43260 markers complete.
43320 markers complete.
43380 markers complete.
43440 markers complete.
43500 markers complete.
43560 markers complete.
43620 markers complete.
43680 markers complete.
43740 markers complete.
43800 markers complete.
43860 markers complete.
43920 markers complete.
43980 markers complete.
44040 markers complete.
44100 markers complete.
44160 markers complete.
44220 markers complete.
44280 markers complete.
44340 markers complete.
44400 markers complete.
44460 markers complete.
44520 markers complete.
44580 markers complete.
44640 markers complete.
44700 markers complete.
44760 markers complete.
44820 markers complete.
44880 markers complete.
44940 markers complete.
45000 markers complete.
45060 markers complete.
45120 markers complete.
45180 markers complete.
45240 markers complete.
45300 markers complete.
45360 markers complete.
45420 markers complete.
45480 markers complete.
45540 markers complete.
45600 markers complete.
45660 markers complete.
45720 markers complete.
45780 markers complete.
45840 markers complete.
45900 markers complete.
45960 markers complete.
46020 markers complete.
46080 markers complete.
46140 markers complete.
46200 markers complete.
46260 markers complete.
46320 markers complete.
46380 markers complete.
46440 markers complete.
46500 markers complete.
46560 markers complete.
46620 markers complete.
46680 markers complete.
46740 markers complete.
46800 markers complete.
46860 markers complete.
46920 markers complete.
46980 markers complete.
47040 markers complete.
47100 markers complete.
47160 markers complete.
47220 markers complete.
47280 markers complete.
47340 markers complete.
47400 markers complete.
47460 markers complete.
47520 markers complete.
47580 markers complete.
47640 markers complete.
47700 markers complete.
47760 markers complete.
47820 markers complete.
47880 markers complete.
47940 markers complete.
48000 markers complete.
48060 markers complete.
48120 markers complete.
48180 markers complete.
48240 markers complete.
48300 markers complete.
48360 markers complete.
48420 markers complete.
48480 markers complete.
48540 markers complete.
48600 markers complete.
48660 markers complete.
48720 markers complete.
48780 markers complete.
48840 markers complete.
48900 markers complete.
48960 markers complete.
49020 markers complete.
49080 markers complete.
49140 markers complete.
49200 markers complete.
49260 markers complete.
49320 markers complete.
49380 markers complete.
49440 markers complete.
49500 markers complete.
49560 markers complete.
49620 markers complete.
49680 markers complete.
49740 markers complete.
49800 markers complete.
49860 markers complete.
49920 markers complete.
49980 markers complete.
50040 markers complete.
50100 markers complete.
50160 markers complete.
50220 markers complete.
50280 markers complete.
50340 markers complete.
50400 markers complete.
50460 markers complete.
50520 markers complete.
50580 markers complete.
50640 markers complete.
50700 markers complete.
50760 markers complete.
50820 markers complete.
50880 markers complete.
50940 markers complete.
51000 markers complete.
51060 markers complete.
51120 markers complete.
51180 markers complete.
51240 markers complete.
51300 markers complete.
51360 markers complete.
51420 markers complete.
51480 markers complete.
51540 markers complete.
51600 markers complete.
51660 markers complete.
51720 markers complete.
51780 markers complete.
51840 markers complete.
51900 markers complete.
51960 markers complete.
52020 markers complete.
52080 markers complete.
52140 markers complete.
52200 markers complete.
52260 markers complete.
52320 markers complete.
52380 markers complete.
52440 markers complete.
52500 markers complete.
52560 markers complete.
52620 markers complete.
52680 markers complete.
52740 markers complete.
52800 markers complete.
52860 markers complete.
52920 markers complete.
52980 markers complete.
53040 markers complete.
53100 markers complete.
53160 markers complete.
53220 markers complete.
53280 markers complete.
53340 markers complete.
53400 markers complete.
53460 markers complete.
53520 markers complete.
53580 markers complete.
53640 markers complete.
53700 markers complete.
53760 markers complete.
53820 markers complete.
53880 markers complete.
53940 markers complete.
54000 markers complete.
54060 markers complete.
54120 markers complete.
54180 markers complete.
54240 markers complete.
54300 markers complete.
54360 markers complete.
54420 markers complete.
54480 markers complete.
54540 markers complete.
54600 markers complete.
54660 markers complete.
54720 markers complete.
54780 markers complete.
54840 markers complete.
54900 markers complete.
54960 markers complete.
55020 markers complete.
55080 markers complete.
55140 markers complete.
55200 markers complete.
55260 markers complete.
55320 markers complete.
55380 markers complete.
55440 markers complete.
55500 markers complete.
55560 markers complete.
55620 markers complete.
55680 markers complete.
55740 markers complete.
55800 markers complete.
55860 markers complete.
55920 markers complete.
55980 markers complete.
56040 markers complete.
56100 markers complete.
56160 markers complete.
56220 markers complete.
56280 markers complete.
56340 markers complete.
56400 markers complete.
56460 markers complete.
56520 markers complete.
56580 markers complete.
56640 markers complete.
56700 markers complete.
56760 markers complete.
56820 markers complete.
56880 markers complete.
56940 markers complete.
57000 markers complete.
57060 markers complete.
57120 markers complete.
57180 markers complete.
57240 markers complete.
57300 markers complete.
57360 markers complete.
57420 markers complete.
57480 markers complete.
57540 markers complete.
57600 markers complete.
57660 markers complete.
57720 markers complete.
57780 markers complete.
57840 markers complete.
57900 markers complete.
57960 markers complete.
58020 markers complete.
58080 markers complete.
58140 markers complete.
58200 markers complete.
58260 markers complete.
58320 markers complete.
58380 markers complete.
58440 markers complete.
58500 markers complete.
58560 markers complete.
58620 markers complete.
58680 markers complete.
58740 markers complete.
58800 markers complete.
58860 markers complete.
58920 markers complete.
58980 markers complete.
59040 markers complete.
59100 markers complete.
59160 markers complete.
59220 markers complete.
59280 markers complete.
59340 markers complete.
59400 markers complete.
59460 markers complete.
59520 markers complete.
59580 markers complete.
59640 markers complete.
59700 markers complete.
59760 markers complete.
59820 markers complete.
59880 markers complete.
59940 markers complete.
60000 markers complete.
60060 markers complete.
60120 markers complete.
60180 markers complete.
60240 markers complete.
60300 markers complete.
60360 markers complete.
60420 markers complete.
60480 markers complete.
60540 markers complete.
60600 markers complete.
60660 markers complete.
60720 markers complete.
60780 markers complete.
60840 markers complete.
60900 markers complete.
60960 markers complete.
61020 markers complete.
61080 markers complete.
61140 markers complete.
61200 markers complete.
61260 markers complete.
61320 markers complete.
61380 markers complete.
61440 markers complete.
61500 markers complete.
61560 markers complete.
61620 markers complete.
61680 markers complete.
61740 markers complete.
61800 markers complete.
61860 markers complete.
61920 markers complete.
61980 markers complete.
62040 markers complete.
62100 markers complete.
62160 markers complete.
62220 markers complete.
62280 markers complete.
62340 markers complete.
62400 markers complete.
62460 markers complete.
62520 markers complete.
62580 markers complete.
62640 markers complete.
62700 markers complete.
62760 markers complete.
62820 markers complete.
62880 markers complete.
62940 markers complete.
63000 markers complete.
63060 markers complete.
63120 markers complete.
63180 markers complete.
63240 markers complete.
63300 markers complete.
63360 markers complete.
63420 markers complete.
63480 markers complete.
63540 markers complete.
63600 markers complete.
63660 markers complete.
63720 markers complete.
63780 markers complete.
63840 markers complete.
63900 markers complete.
63960 markers complete.
64020 markers complete.
64080 markers complete.
64140 markers complete.
64200 markers complete.
64260 markers complete.
64320 markers complete.
64380 markers complete.
64440 markers complete.
64500 markers complete.
64560 markers complete.
64620 markers complete.
64680 markers complete.
64740 markers complete.
64800 markers complete.
64860 markers complete.
64920 markers complete.
64980 markers complete.
65040 markers complete.
65100 markers complete.
65160 markers complete.
65220 markers complete.
65280 markers complete.
65340 markers complete.
65400 markers complete.
65460 markers complete.
65520 markers complete.
65580 markers complete.
65640 markers complete.
65700 markers complete.
65760 markers complete.
65820 markers complete.
65880 markers complete.
65940 markers complete.
66000 markers complete.
66060 markers complete.
66120 markers complete.
66180 markers complete.
66240 markers complete.
66300 markers complete.
66360 markers complete.
66420 markers complete.
66480 markers complete.
66540 markers complete.
66600 markers complete.
66660 markers complete.
66720 markers complete.
66780 markers complete.
66840 markers complete.
66900 markers complete.
66960 markers complete.
67020 markers complete.
67080 markers complete.
67140 markers complete.
67200 markers complete.
67260 markers complete.
67320 markers complete.
67380 markers complete.
67440 markers complete.
67500 markers complete.
67560 markers complete.
67620 markers complete.
67680 markers complete.
67740 markers complete.
67800 markers complete.
67860 markers complete.
67920 markers complete.
67980 markers complete.
68040 markers complete.
68100 markers complete.
68160 markers complete.
68220 markers complete.
68280 markers complete.
68340 markers complete.
68400 markers complete.
68460 markers complete.
68520 markers complete.
68580 markers complete.
68640 markers complete.
68700 markers complete.
68760 markers complete.
68820 markers complete.
68880 markers complete.
68940 markers complete.
69000 markers complete.
69060 markers complete.
69120 markers complete.
69180 markers complete.
69240 markers complete.
69300 markers complete.
69360 markers complete.
69420 markers complete.
69480 markers complete.
69540 markers complete.
69600 markers complete.
69660 markers complete.
69720 markers complete.
69780 markers complete.
69840 markers complete.
69900 markers complete.
69960 markers complete.
70020 markers complete.
70080 markers complete.
70140 markers complete.
70200 markers complete.
70260 markers complete.
70320 markers complete.
70380 markers complete.
70440 markers complete.
70500 markers complete.
70560 markers complete.
70620 markers complete.
70680 markers complete.
70740 markers complete.
70800 markers complete.
70860 markers complete.
70920 markers complete.
70980 markers complete.
71040 markers complete.
71100 markers complete.
71160 markers complete.
71220 markers complete.
71280 markers complete.
71340 markers complete.
71400 markers complete.
71460 markers complete.
71520 markers complete.
71580 markers complete.
71640 markers complete.
71700 markers complete.
71760 markers complete.
71820 markers complete.
71880 markers complete.
71940 markers complete.
72000 markers complete.
72060 markers complete.
72120 markers complete.
72180 markers complete.
72240 markers complete.
72300 markers complete.
72360 markers complete.
72420 markers complete.
72480 markers complete.
72540 markers complete.
72600 markers complete.
72660 markers complete.
72720 markers complete.
72780 markers complete.
72840 markers complete.
72900 markers complete.
72960 markers complete.
73020 markers complete.
73080 markers complete.
73140 markers complete.
73200 markers complete.
73260 markers complete.
73320 markers complete.
73380 markers complete.
73440 markers complete.
73500 markers complete.
73560 markers complete.
73620 markers complete.
73680 markers complete.
73740 markers complete.
73800 markers complete.
73860 markers complete.
73920 markers complete.
73980 markers complete.
74040 markers complete.
74100 markers complete.
74160 markers complete.
74220 markers complete.
74280 markers complete.
74340 markers complete.
74400 markers complete.
74460 markers complete.
74520 markers complete.
74580 markers complete.
74640 markers complete.
74700 markers complete.
74760 markers complete.
74820 markers complete.
74880 markers complete.
74940 markers complete.
75000 markers complete.
75060 markers complete.
75120 markers complete.
75180 markers complete.
75240 markers complete.
75300 markers complete.
75360 markers complete.
75420 markers complete.
75480 markers complete.
75540 markers complete.
75600 markers complete.
75660 markers complete.
75720 markers complete.
75780 markers complete.
75840 markers complete.
75900 markers complete.
75960 markers complete.
76020 markers complete.
76080 markers complete.
76140 markers complete.
76200 markers complete.
76260 markers complete.
76320 markers complete.
76380 markers complete.
76440 markers complete.
76500 markers complete.
76560 markers complete.
76620 markers complete.
76680 markers complete.
76740 markers complete.
76800 markers complete.
76860 markers complete.
76920 markers complete.
76980 markers complete.
77040 markers complete.
77100 markers complete.
77160 markers complete.
77220 markers complete.
77280 markers complete.
77340 markers complete.
77400 markers complete.
77460 markers complete.
77520 markers complete.
77580 markers complete.
77640 markers complete.
77700 markers complete.
77760 markers complete.
77820 markers complete.
77880 markers complete.
77940 markers complete.
77998 markers complete.
Relationship matrix calculation complete.
--pca: Results saved to SampleData1/Fold_0/train_data.eigenval and
SampleData1/Fold_0/train_data.eigenvec .
Deleted directory: SampleData1/Fold_0/hail.train.1.mt
Deleted directory: SampleData1/Fold_0/hail.train.2.mt
Deleted directory: SampleData1/Fold_0/hail.train.3.mt
Deleted directory: SampleData1/Fold_0/hail.train.4.mt
Deleted directory: SampleData1/Fold_0/hail.train.5.mt
Deleted directory: SampleData1/Fold_0/hail.train.6.mt
Deleted directory: SampleData1/Fold_0/hail.train.7.mt
Deleted directory: SampleData1/Fold_0/hail.train.8.mt
Deleted directory: SampleData1/Fold_0/hail.train.9.mt
Deleted directory: SampleData1/Fold_0/hail.train.10.mt
Deleted directory: SampleData1/Fold_0/hail.train.11.mt
Deleted directory: SampleData1/Fold_0/hail.train.12.mt
Deleted directory: SampleData1/Fold_0/hail.train.13.mt
Deleted directory: SampleData1/Fold_0/hail.train.14.mt
Deleted directory: SampleData1/Fold_0/hail.train.15.mt
Deleted directory: SampleData1/Fold_0/hail.train.16.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.1.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 1
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.1
63761 MB RAM detected; reserving 31880 MB for main workspace.
6247 out of 77998 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999944.
6247 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.1.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.1.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.1.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.1.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.1
--chr 1
--out SampleData1/Fold_0/hail.train.1
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
6247 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999944.
6247 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.1.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 01:49 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.1.vcf
ref=ALL.chr1.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr1.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.1
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [1:10177-18871145]
Reference markers: 579,450
Study markers: 554
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 1 second
Estimated ne: 7735
Estimated err: 6.6e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 33 seconds
Window 2 [1:18270328-53836747]
Reference markers: 1,004,286
Study markers: 721
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 6790
Estimated err: 8.6e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 22 seconds
Window 3 [1:49070327-87760656]
Reference markers: 1,094,455
Study markers: 758
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 7514
Estimated err: 7.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 28 seconds
Window 4 [1:85578740-144962397]
Reference markers: 1,037,157
Study markers: 705
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 2 seconds
Estimated ne: 6710
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 3 minutes 29 seconds
Window 5 [1:144224289-175819109]
Reference markers: 830,857
Study markers: 578
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 7663
Estimated err: 8.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 1 minute 15 seconds
Window 6 [1:173093312-213583529]
Reference markers: 1,151,258
Study markers: 766
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 7897
Estimated err: 7.5e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 20 seconds
Window 7 [1:211080854-240239399]
Reference markers: 854,129
Study markers: 715
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 5728
Estimated err: 7.5e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 15 seconds
Window 8 [1:239251841-249240543]
Reference markers: 306,271
Study markers: 296
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 32903
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Initializing Hail with default parameters...
Imputation time: 9 seconds
Cumulative Statistics:
Reference markers: 6,468,094
Study markers: 4,870
Haplotype phasing time: 1 minute 34 seconds
Imputation time: 6 minutes 50 seconds
Total time: 9 minutes 13 seconds
End time: 01:58 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.1.vcf ref=ALL.chr1.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr1.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.1
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Running on Apache Spark version 3.5.1
SparkUI available at http://login02:4040
Welcome to
__ __ <>__
/ /_/ /__ __/ /
/ __ / _ `/ / /
/_/ /_/\_,_/_/_/ version 0.2.132-678e1f52b999
LOGGING: writing to /data/ascher01/uqmmune1/BenchmarkingPGSTools/hail-20240927-1358-0.2.132-678e1f52b999.log
2024-09-27 13:58:45.333 Hail: INFO: scanning VCF for sortedness...
SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder".
SLF4J: Defaulting to no-operation MDCAdapter implementation.
SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details.
2024-09-27 13:59:05.274 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 14:04:57.046 Hail: INFO: wrote matrix table with 6468094 rows and 380 columns in 2 partitions to SampleData1/Fold_0/hail.train.1.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.2.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 2
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.2
63761 MB RAM detected; reserving 31880 MB for main workspace.
8998 out of 114118 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999929.
8998 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.2.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.2.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.2.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.2.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.2
--chr 2
--out SampleData1/Fold_0/hail.train.2
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8998 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999929.
8998 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.2.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 02:04 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.2.vcf
ref=ALL.chr2.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr2.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.2
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [2:10179-18145025]
Reference markers: 573,772
Study markers: 880
Burnin iteration 1: 5 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 7544
Estimated err: 1.1e-03
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 25 seconds
Window 2 [2:16417586-49114274]
Reference markers: 1,053,726
Study markers: 1,173
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 3 seconds
Estimated ne: 10035
Estimated err: 8.6e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 32 seconds
Window 3 [2:47323299-88186309]
Reference markers: 1,255,428
Study markers: 1,184
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 3 seconds
Estimated ne: 8361
Estimated err: 8.7e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 33 seconds
Window 4 [2:86643697-134624574]
Reference markers: 1,221,295
Study markers: 1,092
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 8660
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 45 seconds
Window 5 [2:133390491-177333395]
Reference markers: 1,235,397
Study markers: 1,202
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 8841
Estimated err: 8.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 2 seconds
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 34 seconds
Window 6 [2:175084768-220694589]
Reference markers: 1,278,696
Study markers: 1,258
Burnin iteration 1: 3 seconds
Burnin iteration 2: 3 seconds
Burnin iteration 3: 3 seconds
Estimated ne: 9252
Estimated err: 2.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 2 seconds
Phasing iteration 6: 2 seconds
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 2 seconds
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 35 seconds
Window 7 [2:218829611-242575162]
Reference markers: 734,807
Study markers: 1,103
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9018
Estimated err: 8.8e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 16 seconds
Window 8 [2:241170836-243188367]
Reference markers: 65,148
Study markers: 65
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 245134
Estimated err: 1.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Cumulative Statistics:
Reference markers: 7,081,600
Study markers: 7,568
Haplotype phasing time: 2 minutes 22 seconds
Imputation time: 3 minutes 44 seconds
Total time: 7 minutes 49 seconds
End time: 02:12 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.2.vcf ref=ALL.chr2.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr2.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.2
2024-09-27 14:12:47.026 Hail: INFO: scanning VCF for sortedness...
2024-09-27 14:13:05.331 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 14:18:18.220 Hail: INFO: wrote matrix table with 7081600 rows and 380 columns in 2 partitions to SampleData1/Fold_0/hail.train.2.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.3.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 3
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.3
63761 MB RAM detected; reserving 31880 MB for main workspace.
7649 out of 114118 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999916.
7649 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.3.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.3.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.3.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.3.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.3
--chr 3
--out SampleData1/Fold_0/hail.train.3
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
7649 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999916.
7649 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.3.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 02:18 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.3.vcf
ref=ALL.chr3.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr3.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.3
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [3:60069-21643990]
Reference markers: 718,343
Study markers: 1,014
Burnin iteration 1: 5 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 7964
Estimated err: 9.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 29 seconds
Window 2 [3:19789059-60278775]
Reference markers: 1,194,873
Study markers: 1,173
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 8022
Estimated err: 9.3e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 37 seconds
Window 3 [3:59462914-105372608]
Reference markers: 1,325,897
Study markers: 1,253
Burnin iteration 1: 4 seconds
Burnin iteration 2: 3 seconds
Burnin iteration 3: 3 seconds
Estimated ne: 8492
Estimated err: 8.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 36 seconds
Window 4 [3:103398092-143445761]
Reference markers: 1,153,131
Study markers: 1,258
Burnin iteration 1: 4 seconds
Burnin iteration 2: 3 seconds
Burnin iteration 3: 3 seconds
Estimated ne: 8060
Estimated err: 8.4e-04
Phasing iteration 1: 2 seconds
Phasing iteration 2: 2 seconds
Phasing iteration 3: 2 seconds
Phasing iteration 4: 1 second
Phasing iteration 5: 2 seconds
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 32 seconds
Window 5 [3:141741447-182238285]
Reference markers: 1,155,993
Study markers: 1,208
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 3 seconds
Estimated ne: 7660
Estimated err: 8.4e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 27 seconds
Window 6 [3:179790776-197962381]
Reference markers: 545,816
Study markers: 848
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 8667
Estimated err: 8.6e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 13 seconds
Cumulative Statistics:
Reference markers: 5,832,276
Study markers: 6,419
Haplotype phasing time: 2 minutes 1 second
Imputation time: 2 minutes 53 seconds
Total time: 6 minutes 14 seconds
End time: 02:24 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.3.vcf ref=ALL.chr3.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr3.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.3
2024-09-27 14:24:32.872 Hail: INFO: scanning VCF for sortedness...
2024-09-27 14:24:57.642 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 14:33:22.829 Hail: INFO: wrote matrix table with 5832276 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.3.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.4.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 4
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.4
63761 MB RAM detected; reserving 31880 MB for main workspace.
7312 out of 114118 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999935.
7312 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.4.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.4.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.4.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.4.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.4
--chr 4
--out SampleData1/Fold_0/hail.train.4
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
7312 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999935.
7312 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.4.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 02:33 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.4.vcf
ref=ALL.chr4.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr4.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.4
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [4:10005-22520039]
Reference markers: 786,334
Study markers: 973
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 1 second
Estimated ne: 8066
Estimated err: 9.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 18 seconds
Window 2 [4:20544690-60086601]
Reference markers: 1,127,621
Study markers: 1,186
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 1 second
Estimated ne: 9301
Estimated err: 8.4e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 19 seconds
Window 3 [4:57715343-105764064]
Reference markers: 1,443,530
Study markers: 1,220
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 8989
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 23 seconds
Window 4 [4:102709973-150650017]
Reference markers: 1,376,146
Study markers: 1,281
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9225
Estimated err: 8.8e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 22 seconds
Window 5 [4:148086441-182155049]
Reference markers: 1,012,842
Study markers: 1,191
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 8541
Estimated err: 7.8e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 13 seconds
Window 6 [4:181312927-191043881]
Reference markers: 316,589
Study markers: 612
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 10400
Estimated err: 9.5e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 6 seconds
Cumulative Statistics:
Reference markers: 5,732,585
Study markers: 6,121
Haplotype phasing time: 1 minute 13 seconds
Imputation time: 1 minute 41 seconds
Total time: 4 minutes 6 seconds
End time: 02:37 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.4.vcf ref=ALL.chr4.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr4.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.4
2024-09-27 14:37:29.240 Hail: INFO: scanning VCF for sortedness...
2024-09-27 14:37:52.712 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 14:45:55.265 Hail: INFO: wrote matrix table with 5732585 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.4.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.5.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 5
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.5
63761 MB RAM detected; reserving 31880 MB for main workspace.
10632 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999882.
10632 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.5.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.5.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.5.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.5.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.5
--chr 5
--out SampleData1/Fold_0/hail.train.5
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
10632 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999882.
10632 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.5.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 02:45 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.5.vcf
ref=ALL.chr5.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr5.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.5
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [5:10043-24839896]
Reference markers: 785,833
Study markers: 1,656
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 11105
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 25 seconds
Window 2 [5:22775078-69306446]
Reference markers: 1,297,643
Study markers: 1,974
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 19142
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 29 seconds
Window 3 [5:67534999-111982027]
Reference markers: 1,223,992
Study markers: 2,012
Burnin iteration 1: 3 seconds
Burnin iteration 2: 3 seconds
Estimated ne: 19277
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 30 seconds
Window 4 [5:109533719-149269607]
Reference markers: 1,201,670
Study markers: 1,968
Burnin iteration 1: 2 seconds
Burnin iteration 2: 3 seconds
Estimated ne: 18121
Estimated err: 1.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 30 seconds
Window 5 [5:148226432-174766607]
Reference markers: 787,384
Study markers: 1,787
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 12549
Estimated err: 2.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 15 seconds
Window 6 [5:174167120-180904689]
Reference markers: 201,966
Study markers: 480
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 7643
Estimated err: 9.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Cumulative Statistics:
Reference markers: 5,265,763
Study markers: 9,450
Haplotype phasing time: 1 minute 33 seconds
Imputation time: 2 minutes 13 seconds
Total time: 4 minutes 54 seconds
End time: 02:50 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.5.vcf ref=ALL.chr5.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr5.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.5
2024-09-27 14:50:49.927 Hail: INFO: scanning VCF for sortedness...
2024-09-27 14:51:13.675 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 14:59:00.308 Hail: INFO: wrote matrix table with 5265763 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.5.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.6.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 6
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.6
63761 MB RAM detected; reserving 31880 MB for main workspace.
10068 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999902.
10068 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.6.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.6.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.6.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.6.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.6
--chr 6
--out SampleData1/Fold_0/hail.train.6
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
10068 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999902.
10068 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.6.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 02:59 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.6.vcf
ref=ALL.chr6.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr6.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.6
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [6:63854-21294216]
Reference markers: 658,963
Study markers: 1,608
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 11789
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 24 seconds
Window 2 [6:20005714-56895330]
Reference markers: 1,115,840
Study markers: 1,961
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 15522
Estimated err: 6.4e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 25 seconds
Window 3 [6:53556519-110284141]
Reference markers: 1,605,180
Study markers: 2,128
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 18780
Estimated err: 1.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 31 seconds
Window 4 [6:108297729-149276979]
Reference markers: 1,157,048
Study markers: 2,014
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 19637
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 22 seconds
Window 5 [6:148481212-170715177]
Reference markers: 694,787
Study markers: 1,580
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 11587
Estimated err: 2.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 12 seconds
Window 6 [6:169508794-171053406]
Reference markers: 46,342
Study markers: 86
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 209543
Estimated err: 5.0e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 1 second
Cumulative Statistics:
Reference markers: 5,024,119
Study markers: 8,904
Haplotype phasing time: 1 minute 17 seconds
Imputation time: 1 minute 55 seconds
Total time: 4 minutes 2 seconds
End time: 03:03 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.6.vcf ref=ALL.chr6.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr6.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.6
2024-09-27 15:03:03.199 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:03:23.648 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 15:10:43.593 Hail: INFO: wrote matrix table with 5024119 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.6.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.7.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 7
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.7
63761 MB RAM detected; reserving 31880 MB for main workspace.
9496 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999897.
9496 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.7.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.7.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.7.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.7.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.7
--chr 7
--out SampleData1/Fold_0/hail.train.7
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
9496 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999897.
9496 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.7.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 03:10 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.7.vcf
ref=ALL.chr7.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr7.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.7
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [7:14808-22507950]
Reference markers: 834,932
Study markers: 1,527
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 10288
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 23 seconds
Window 2 [7:21712814-53284120]
Reference markers: 941,559
Study markers: 1,889
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 20441
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 22 seconds
Window 3 [7:51371519-102207161]
Reference markers: 1,373,374
Study markers: 1,945
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 17409
Estimated err: 6.5e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 28 seconds
Window 4 [7:98972040-141848611]
Reference markers: 1,207,792
Study markers: 2,088
Burnin iteration 1: 2 seconds
Burnin iteration 2: 3 seconds
Estimated ne: 19500
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 21 seconds
Window 5 [7:140215356-159128567]
Reference markers: 589,076
Study markers: 1,342
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 8786
Estimated err: 9.1e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 11 seconds
Cumulative Statistics:
Reference markers: 4,716,715
Study markers: 8,421
Haplotype phasing time: 1 minute 17 seconds
Imputation time: 1 minute 45 seconds
Total time: 3 minutes 42 seconds
End time: 03:14 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.7.vcf ref=ALL.chr7.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr7.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.7
2024-09-27 15:14:26.167 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:14:46.616 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 15:21:32.811 Hail: INFO: wrote matrix table with 4716715 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.7.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.8.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 8
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.8
63761 MB RAM detected; reserving 31880 MB for main workspace.
8867 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8867 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.8.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.8.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.8.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.8.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.8
--chr 8
--out SampleData1/Fold_0/hail.train.8
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8867 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8867 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.8.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 03:21 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.8.vcf
ref=ALL.chr8.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr8.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.8
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [8:11740-18799855]
Reference markers: 877,773
Study markers: 1,473
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 9532
Estimated err: 2.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 24 seconds
Window 2 [8:17678413-59729351]
Reference markers: 1,232,933
Study markers: 1,985
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 20310
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 27 seconds
java.lang.IllegalArgumentException: Duplicate marker: 8 89329148 . G A
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.8.vcf ref=ALL.chr8.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr8.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.8
2024-09-27 15:24:02.637 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:24:11.134 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 15:27:05.495 Hail: INFO: wrote matrix table with 2039669 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.8.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.9.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 9
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.9
63761 MB RAM detected; reserving 31880 MB for main workspace.
7768 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999892.
7768 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.9.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.9.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.9.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.9.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.9
--chr 9
--out SampleData1/Fold_0/hail.train.9
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
7768 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999892.
7768 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.9.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 03:27 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.9.vcf
ref=ALL.chr9.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr9.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.9
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [9:10163-19603557]
Reference markers: 766,446
Study markers: 1,563
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 11323
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 24 seconds
Window 2 [9:18636347-80220415]
Reference markers: 1,009,966
Study markers: 1,724
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 16522
Estimated err: 1.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 27 seconds
Window 3 [9:79151500-112369214]
Reference markers: 978,438
Study markers: 1,899
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 18394
Estimated err: 2.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 24 seconds
Window 4 [9:110879571-137041122]
Reference markers: 775,425
Study markers: 1,621
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 1 second
Estimated ne: 10878
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 13 seconds
Window 5 [9:136539237-141148854]
Reference markers: 166,298
Study markers: 363
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 13390
Estimated err: 8.8e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 21 seconds
Cumulative Statistics:
Reference markers: 3,560,687
Study markers: 6,877
Haplotype phasing time: 1 minute 5 seconds
Imputation time: 1 minute 49 seconds
Total time: 3 minutes 31 seconds
End time: 03:30 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.9.vcf ref=ALL.chr9.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr9.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.9
2024-09-27 15:30:37.280 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:30:52.044 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 15:35:56.524 Hail: INFO: wrote matrix table with 3560687 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.9.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.10.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 10
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.10
63761 MB RAM detected; reserving 31880 MB for main workspace.
8824 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
8824 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.10.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.10.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.10.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.10.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.10
--chr 10
--out SampleData1/Fold_0/hail.train.10
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8824 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
8824 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.10.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 03:35 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.10.vcf
ref=ALL.chr10.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr10.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.10
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [10:60494-18538847]
Reference markers: 622,290
Study markers: 1,543
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 10772
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 21 seconds
Window 2 [10:17068251-54798179]
Reference markers: 1,000,400
Study markers: 1,560
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 16442
Estimated err: 8.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 15 seconds
Window 3 [10:53792357-90734239]
Reference markers: 1,092,804
Study markers: 1,942
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 17386
Estimated err: 7.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 0 seconds
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 14 seconds
Window 4 [10:88784362-123900556]
Reference markers: 1,013,192
Study markers: 1,904
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 21656
Estimated err: 2.4e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 0 seconds
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 10 seconds
Window 5 [10:123093921-135524373]
Reference markers: 406,013
Study markers: 1,118
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 10684
Estimated err: 9.8e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Cumulative Statistics:
Reference markers: 3,992,219
Study markers: 7,763
Haplotype phasing time: 49 seconds
Imputation time: 1 minute 5 seconds
Total time: 2 minutes 29 seconds
End time: 03:38 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.10.vcf ref=ALL.chr10.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr10.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.10
2024-09-27 15:38:26.409 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:38:42.411 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 15:44:28.695 Hail: INFO: wrote matrix table with 3992219 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.10.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.11.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 11
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.11
63761 MB RAM detected; reserving 31880 MB for main workspace.
8420 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.9999.
8420 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.11.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.11.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.11.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.11.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.11
--chr 11
--out SampleData1/Fold_0/hail.train.11
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8420 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.9999.
8420 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.11.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 03:44 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.11.vcf
ref=ALL.chr11.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr11.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.11
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [11:61395-22535739]
Reference markers: 711,691
Study markers: 1,629
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 11525
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 22 seconds
Window 2 [11:21349458-70803324]
Reference markers: 1,428,012
Study markers: 2,035
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 18794
Estimated err: 1.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 0 seconds
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 33 seconds
Window 3 [11:69604836-112290497]
Reference markers: 1,283,964
Study markers: 2,125
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 21225
Estimated err: 1.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 25 seconds
Window 4 [11:110168424-133043804]
Reference markers: 692,681
Study markers: 1,740
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 18833
Estimated err: 2.1e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 1 second
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 1 second
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 12 seconds
Window 5 [11:132350788-134946453]
Reference markers: 88,218
Study markers: 273
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 30941
Estimated err: 7.5e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 1 second
Cumulative Statistics:
Reference markers: 4,045,628
Study markers: 7,460
Haplotype phasing time: 54 seconds
Imputation time: 1 minute 34 seconds
Total time: 3 minutes 23 seconds
End time: 03:47 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.11.vcf ref=ALL.chr11.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr11.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.11
2024-09-27 15:47:52.751 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:48:09.331 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 15:53:57.158 Hail: INFO: wrote matrix table with 4045628 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.11.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.12.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 12
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.12
63761 MB RAM detected; reserving 31880 MB for main workspace.
8198 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8198 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.12.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.12.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.12.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.12.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.12
--chr 12
--out SampleData1/Fold_0/hail.train.12
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8198 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8198 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.12.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 03:53 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.12.vcf
ref=ALL.chr12.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr12.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.12
nthreads=8
java.lang.IllegalArgumentException: Duplicate marker: 12 8400000 . T G
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.12.vcf ref=ALL.chr12.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr12.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.12
2024-09-27 15:54:16.909 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:54:17.167 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 15:54:17.658 Hail: INFO: wrote matrix table with 0 rows and 380 columns in 0 partitions to SampleData1/Fold_0/hail.train.12.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.13.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 13
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.13
63761 MB RAM detected; reserving 31880 MB for main workspace.
6350 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
6350 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.13.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.13.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.13.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.13.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.13
--chr 13
--out SampleData1/Fold_0/hail.train.13
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
6350 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
6350 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.13.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 03:54 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.13.vcf
ref=ALL.chr13.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr13.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.13
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [13:19020047-40892639]
Reference markers: 671,822
Study markers: 1,483
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 9869
Estimated err: 2.2e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 20 seconds
Window 2 [13:39849087-81726083]
Reference markers: 1,198,543
Study markers: 1,992
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 19344
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 25 seconds
Window 3 [13:79491658-110702359]
Reference markers: 937,844
Study markers: 1,855
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 12534
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 0 seconds
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 0 seconds
Phasing iteration 7: 1 second
Phasing iteration 8: 0 seconds
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 14 seconds
Window 4 [13:109580155-115109852]
Reference markers: 180,096
Study markers: 461
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 8263
Estimated err: 1.3e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Cumulative Statistics:
Reference markers: 2,857,916
Study markers: 5,502
Haplotype phasing time: 49 seconds
Imputation time: 1 minute 2 seconds
Total time: 2 minutes 22 seconds
End time: 03:56 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.13.vcf ref=ALL.chr13.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr13.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.13
2024-09-27 15:56:40.644 Hail: INFO: scanning VCF for sortedness...
2024-09-27 15:56:53.103 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:01:03.478 Hail: INFO: wrote matrix table with 2857916 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.13.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.14.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 14
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.14
63761 MB RAM detected; reserving 31880 MB for main workspace.
5742 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999852.
5742 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.14.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.14.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.14.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.14.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.14
--chr 14
--out SampleData1/Fold_0/hail.train.14
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5742 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999852.
5742 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.14.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:01 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.14.vcf
ref=ALL.chr14.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr14.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.14
nthreads=8
java.lang.IllegalArgumentException: Duplicate marker: 14 21649957 . C T
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.14.vcf ref=ALL.chr14.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr14.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.14
2024-09-27 16:01:21.924 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:01:22.171 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 16:01:22.527 Hail: INFO: wrote matrix table with 0 rows and 380 columns in 0 partitions to SampleData1/Fold_0/hail.train.14.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.15.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 15
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.15
63761 MB RAM detected; reserving 31880 MB for main workspace.
5569 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
5569 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.15.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.15.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.15.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.15.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.15
--chr 15
--out SampleData1/Fold_0/hail.train.15
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5569 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
5569 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.15.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:01 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.15.vcf
ref=ALL.chr15.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr15.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.15
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [15:20000041-32890838]
Reference markers: 325,223
Study markers: 628
Burnin iteration 1: 3 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 6297
Estimated err: 3.1e-03
Phasing iteration 1: 1 second
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 2 minutes 42 seconds
Window 2 [15:32564155-62934668]
Reference markers: 905,216
Study markers: 1,635
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 18800
Estimated err: 2.3e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 13 seconds
Window 3 [15:61592413-93656456]
Reference markers: 943,933
Study markers: 1,797
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 19673
Estimated err: 2.1e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 1 second
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 1 second
Phasing iteration 12: 0 seconds
Imputation time: 14 seconds
Window 4 [15:92986651-102521131]
Reference markers: 318,635
Study markers: 976
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 10402
Estimated err: 1.0e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Cumulative Statistics:
Reference markers: 2,424,689
Study markers: 4,877
Haplotype phasing time: 38 seconds
Imputation time: 3 minutes 17 seconds
Total time: 4 minutes 7 seconds
End time: 04:05 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.15.vcf ref=ALL.chr15.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr15.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.15
2024-09-27 16:05:29.868 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:05:40.131 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:09:07.754 Hail: INFO: wrote matrix table with 2424689 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.15.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.16.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 16
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.16
63761 MB RAM detected; reserving 31880 MB for main workspace.
6069 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99988.
6069 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.16.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.16.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.16.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.16.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.16
--chr 16
--out SampleData1/Fold_0/hail.train.16
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
6069 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99988.
6069 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.16.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:09 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.16.vcf
ref=ALL.chr16.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr16.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.16
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [16:60086-19243846]
Reference markers: 737,912
Study markers: 1,487
Burnin iteration 1: 5 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 9948
Estimated err: 9.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 22 seconds
Window 2 [16:18062715-58867225]
Reference markers: 828,530
Study markers: 1,630
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 12060
Estimated err: 2.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
java.util.zip.ZipException: invalid distance too far back
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:165)
at java.base/java.util.zip.GZIPInputStream.read(GZIPInputStream.java:118)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
at blbutil.BGZipIt.inflateBlock(BGZipIt.java:314)
at blbutil.BGZipIt.lambda$readAndInflateBlocks$3(BGZipIt.java:263)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.Nodes$SizedCollectorTask.compute(Nodes.java:1886)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Terminating program.
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.16.vcf ref=ALL.chr16.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr16.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.16
2024-09-27 16:10:30.430 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:10:33.858 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:11:36.417 Hail: INFO: wrote matrix table with 725178 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.16.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.17.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 17
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.17
63761 MB RAM detected; reserving 31880 MB for main workspace.
5723 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999884.
5723 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.17.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.17.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.17.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.17.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.17
--chr 17
--out SampleData1/Fold_0/hail.train.17
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5723 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999884.
5723 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.17.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:11 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.17.vcf
ref=ALL.chr17.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr17.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.17
nthreads=8
java.lang.IllegalArgumentException: Duplicate marker: 17 1144632 . C CT
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 16:11:51.161 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.17.vcf ref=ALL.chr17.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr17.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.17
2024-09-27 16:11:51.378 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 16:11:52.022 Hail: INFO: wrote matrix table with 0 rows and 380 columns in 0 partitions to SampleData1/Fold_0/hail.train.17.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.18.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 18
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.18
63761 MB RAM detected; reserving 31880 MB for main workspace.
5578 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
5578 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.18.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.18.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.18.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.18.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.18
--chr 18
--out SampleData1/Fold_0/hail.train.18
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5578 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
5578 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.18.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:11 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.18.vcf
ref=ALL.chr18.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr18.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.18
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [18:10083-15185474]
Reference markers: 478,762
Study markers: 1,287
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 1 second
Estimated ne: 9156
Estimated err: 8.7e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 19 seconds
Window 2 [18:13460439-56841603]
Reference markers: 1,173,645
Study markers: 2,085
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 17072
Estimated err: 6.9e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 22 seconds
Window 3 [18:55786342-76632429]
Reference markers: 654,683
Study markers: 1,587
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 8853
Estimated err: 2.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 1 second
Phasing iteration 3: 0 seconds
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 0 seconds
Phasing iteration 9: 1 second
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 11 seconds
Window 4 [18:75886666-78017156]
Reference markers: 77,128
Study markers: 181
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 92867
Estimated err: 3.4e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 1 second
Cumulative Statistics:
Reference markers: 2,267,185
Study markers: 4,912
Haplotype phasing time: 43 seconds
Imputation time: 53 seconds
Total time: 2 minutes 5 seconds
End time: 04:13 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.18.vcf ref=ALL.chr18.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr18.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.18
2024-09-27 16:13:57.798 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:14:07.440 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:17:28.490 Hail: INFO: wrote matrix table with 2267185 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.18.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.19.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 19
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.19
63761 MB RAM detected; reserving 31880 MB for main workspace.
4364 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99987.
4364 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.19.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.19.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.19.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.19.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.19
--chr 19
--out SampleData1/Fold_0/hail.train.19
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
4364 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99987.
4364 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.19.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:17 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.19.vcf
ref=ALL.chr19.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr19.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.19
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [19:60842-15654475]
Reference markers: 522,636
Study markers: 1,252
Burnin iteration 1: 4 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 8340
Estimated err: 9.4e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 18 seconds
Window 2 [19:14657353-48467440]
Reference markers: 980,357
Study markers: 1,650
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 18607
Estimated err: 2.0e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 15 seconds
Window 3 [19:47339481-59118924]
Reference markers: 395,275
Study markers: 1,025
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9736
Estimated err: 1.0e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 1 second
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Cumulative Statistics:
Reference markers: 1,832,506
Study markers: 3,788
Haplotype phasing time: 38 seconds
Imputation time: 41 seconds
Total time: 1 minute 45 seconds
End time: 04:19 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.19.vcf ref=ALL.chr19.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr19.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.19
2024-09-27 16:19:13.692 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:19:22.149 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:21:59.026 Hail: INFO: wrote matrix table with 1832506 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.19.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.20.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 20
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.20
63761 MB RAM detected; reserving 31880 MB for main workspace.
4916 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
4916 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.20.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.20.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.20.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.20.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.20
--chr 20
--out SampleData1/Fold_0/hail.train.20
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
4916 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
4916 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.20.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:21 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.20.vcf
ref=ALL.chr20.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr20.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.20
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [20:60343-17038340]
Reference markers: 507,580
Study markers: 1,478
Burnin iteration 1: 5 seconds
Burnin iteration 2: 2 seconds
Burnin iteration 3: 2 seconds
Estimated ne: 9314
Estimated err: 8.3e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 19 seconds
Window 2 [20:16012786-51450735]
Reference markers: 956,124
Study markers: 1,813
Burnin iteration 1: 2 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 18916
Estimated err: 2.1e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 14 seconds
Window 3 [20:50160268-62965354]
Reference markers: 419,983
Study markers: 1,207
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 10473
Estimated err: 2.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Cumulative Statistics:
Reference markers: 1,812,841
Study markers: 4,346
Haplotype phasing time: 40 seconds
Imputation time: 40 seconds
Total time: 1 minute 42 seconds
End time: 04:23 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.20.vcf ref=ALL.chr20.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr20.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.20
2024-09-27 16:23:41.589 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:23:49.312 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:26:24.612 Hail: INFO: wrote matrix table with 1812841 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.20.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.21.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 21
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.21
63761 MB RAM detected; reserving 31880 MB for main workspace.
2811 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
2811 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.21.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.21.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.21.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.21.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.21
--chr 21
--out SampleData1/Fold_0/hail.train.21
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
2811 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
2811 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.21.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:26 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.21.vcf
ref=ALL.chr21.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr21.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.21
nthreads=8
Reference samples: 2,504
Study samples: 380
Window 1 [21:9411239-36416332]
Reference markers: 719,753
Study markers: 1,469
Burnin iteration 1: 3 seconds
Burnin iteration 2: 2 seconds
Estimated ne: 17989
Estimated err: 2.3e-04
Phasing iteration 1: 1 second
Phasing iteration 2: 1 second
Phasing iteration 3: 1 second
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 1 second
Phasing iteration 7: 1 second
Phasing iteration 8: 1 second
Phasing iteration 9: 1 second
Phasing iteration 10: 1 second
Phasing iteration 11: 1 second
Phasing iteration 12: 1 second
Imputation time: 21 seconds
Window 2 [21:35704550-48119740]
Reference markers: 407,769
Study markers: 1,061
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9530
Estimated err: 1.1e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Cumulative Statistics:
Reference markers: 1,105,538
Study markers: 2,458
Haplotype phasing time: 21 seconds
Imputation time: 29 seconds
Total time: 1 minute 16 seconds
End time: 04:27 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.21.vcf ref=ALL.chr21.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr21.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.21
2024-09-27 16:27:40.991 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:27:45.921 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:29:24.159 Hail: INFO: wrote matrix table with 1105538 rows and 380 columns in 1 partition to SampleData1/Fold_0/hail.train.21.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/train_data.QC.clumped.pruned.22.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned
--chr 22
--make-bed
--out SampleData1/Fold_0/train_data.QC.clumped.pruned.22
63761 MB RAM detected; reserving 31880 MB for main workspace.
2831 out of 172878 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999896.
2831 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/train_data.QC.clumped.pruned.22.bed +
SampleData1/Fold_0/train_data.QC.clumped.pruned.22.bim +
SampleData1/Fold_0/train_data.QC.clumped.pruned.22.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.train.22.log.
Options in effect:
--bfile SampleData1/Fold_0/train_data.QC.clumped.pruned.22
--chr 22
--out SampleData1/Fold_0/hail.train.22
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
2831 variants loaded from .bim file.
380 people (183 males, 197 females) loaded from .fam.
380 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 380 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999896.
2831 variants and 380 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.train.22.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:29 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.train.22.vcf
ref=ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr22.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.train.22
nthreads=8
java.util.zip.ZipException: invalid distance too far back
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:165)
at java.base/java.util.zip.GZIPInputStream.read(GZIPInputStream.java:118)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
at blbutil.BGZipIt.inflateBlock(BGZipIt.java:314)
at blbutil.BGZipIt.lambda$readAndInflateBlocks$3(BGZipIt.java:263)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.Nodes$SizedCollectorTask.compute(Nodes.java:1886)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736)
at java.base/java.util.stream.Nodes.collect(Nodes.java:333)
at java.base/java.util.stream.ReferencePipeline.evaluateToNode(ReferencePipeline.java:109)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
at blbutil.BGZipIt.readAndInflateBlocks(BGZipIt.java:264)
at blbutil.BGZipIt.fillBuffer(BGZipIt.java:148)
at blbutil.BGZipIt.next(BGZipIt.java:137)
at blbutil.BGZipIt.next(BGZipIt.java:53)
at blbutil.BlockLineReader.lambda$startFileReadingThread$0(BlockLineReader.java:107)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 16:29:36.872 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.train.22.vcf ref=ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr22.GRCh37.map out=SampleData1/Fold_0/beagle.hail.train.22
2024-09-27 16:29:37.076 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 16:29:37.469 Hail: INFO: wrote matrix table with 0 rows and 380 columns in 0 partitions to SampleData1/Fold_0/hail.train.22.mt
showing top 5 rows
None
1 SampleData1/Fold_0/hail.train.1.mt (6468094, 380) (6468094, 380)
2 SampleData1/Fold_0/hail.train.2.mt (7081600, 380) (13549694, 380)
3 SampleData1/Fold_0/hail.train.3.mt (5832276, 380) (19381970, 380)
4 SampleData1/Fold_0/hail.train.4.mt (5732585, 380) (25114555, 380)
5 SampleData1/Fold_0/hail.train.5.mt (5265763, 380) (30380318, 380)
6 SampleData1/Fold_0/hail.train.6.mt (5024119, 380) (35404437, 380)
7 SampleData1/Fold_0/hail.train.7.mt (4716715, 380) (40121152, 380)
8 SampleData1/Fold_0/hail.train.8.mt (2039669, 380) (42160821, 380)
9 SampleData1/Fold_0/hail.train.9.mt (3560687, 380) (45721508, 380)
10 SampleData1/Fold_0/hail.train.10.mt (3992219, 380) (49713727, 380)
11 SampleData1/Fold_0/hail.train.11.mt (4045628, 380) (53759355, 380)
12 SampleData1/Fold_0/hail.train.12.mt (0, 380) (53759355, 380)
13 SampleData1/Fold_0/hail.train.13.mt (2857916, 380) (56617271, 380)
14 SampleData1/Fold_0/hail.train.14.mt (0, 380) (56617271, 380)
15 SampleData1/Fold_0/hail.train.15.mt (2424689, 380) (59041960, 380)
16 SampleData1/Fold_0/hail.train.16.mt (725178, 380) (59767138, 380)
17 SampleData1/Fold_0/hail.train.17.mt (0, 380) (59767138, 380)
18 SampleData1/Fold_0/hail.train.18.mt (2267185, 380) (62034323, 380)
19 SampleData1/Fold_0/hail.train.19.mt (1832506, 380) (63866829, 380)
20 SampleData1/Fold_0/hail.train.20.mt (1812841, 380) (65679670, 380)
21 SampleData1/Fold_0/hail.train.21.mt (1105538, 380) (66785208, 380)
22 SampleData1/Fold_0/hail.train.22.mt (0, 380) (66785208, 380)
(66785208, 380)
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.util.SizeEstimator$ (file:/data/ascher01/uqmmune1/miniconda3/envs/genetics/lib/python3.10/site-packages/pyspark/jars/spark-core_2.12-3.5.1.jar) to field java.util.concurrent.locks.ReentrantReadWriteLock.readerLock
WARNING: Please consider reporting this to the maintainers of org.apache.spark.util.SizeEstimator$
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2024-09-27 16:34:45.048 Hail: INFO: wrote matrix table with 66785208 rows and 380 columns in 20 partitions to SampleData1/Fold_0/HAILTRAIN.mt
showing top 5 rows
None
2024-09-27 16:34:46.492 Hail: INFO: Reading table to impute column types
2024-09-27 16:34:48.264 Hail: INFO: Finished type imputation (0 + 1) / 1]
Loading field 'chr_name' as type int32 (imputed)
Loading field 'chr_position' as type int32 (imputed)
Loading field 'effect_allele' as type str (imputed)
Loading field 'other_allele' as type str (imputed)
Loading field 'effect_weight' as type float64 (imputed)
2024-09-27 16:34:48.331 Hail: WARN: cols(): Resulting column table is sorted by 'col_key'.
To preserve matrix table column order, first unkey columns with 'key_cols_by()'
2024-09-27 16:36:27.301 Hail: INFO: Ordering unsorted dataset with network shuffle
2024-09-27 16:36:29.066 Hail: INFO: Ordering unsorted dataset with network shuffle
2024-09-27 16:39:14.336 Hail: INFO: Ordering unsorted dataset with network shuffle
2024-09-27 16:48:09.426 Hail: INFO: Coerced sorted dataset=====> (19 + 1) / 20]
2024-09-27 16:48:09.727 Hail: INFO: merging 9 files totalling 10.1K...
2024-09-27 16:48:09.771 Hail: INFO: while writing:
SampleData1/Fold_0/HAIL/train_PRS.txt
merge time: 44.263ms
2024-09-27 16:48:10.122 Hail: INFO: Reading table to impute column types
2024-09-27 16:48:10.372 Hail: INFO: Finished type imputation
Loading field 's' as type str (imputed)
Loading field 'prs' as type float64 (imputed)
2024-09-27 16:48:10.574 Hail: INFO: Coerced sorted dataset
subjectID PGS002724
0 HG00097_HG00097 18.058
1 HG00099_HG00099 -35.684
2 HG00101_HG00101 23.943
3 HG00102_HG00102 -20.056
4 HG00103_HG00103 17.401
subjectID PGS002724
0 HG00097_HG00097 18.058
1 HG00099_HG00099 -35.684
2 HG00101_HG00101 23.943
3 HG00102_HG00102 -20.056
4 HG00103_HG00103 17.401
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.1.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 1
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.1
63761 MB RAM detected; reserving 31880 MB for main workspace.
14013 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999881.
14013 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.1.bed +
SampleData1/Fold_0/test_data.clumped.pruned.1.bim +
SampleData1/Fold_0/test_data.clumped.pruned.1.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.1.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.1
--chr 1
--out SampleData1/Fold_0/hail.test.1
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
14013 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999881.
14013 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.1.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:48 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.1.vcf
ref=ALL.chr1.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr1.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.1
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [1:10177-18871145]
Reference markers: 579,450
Study markers: 1,344
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 12600
Estimated err: 2.5e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 2 [1:18270328-53836747]
Reference markers: 1,004,286
Study markers: 1,861
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 12483
Estimated err: 8.0e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 3 [1:49070327-87760656]
Reference markers: 1,094,455
Study markers: 1,963
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 15371
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 1 second
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 4 [1:85578740-144962397]
Reference markers: 1,037,157
Study markers: 1,824
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16088
Estimated err: 8.7e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 18 seconds
Window 5 [1:144224289-175819109]
Reference markers: 830,857
Study markers: 1,537
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 14501
Estimated err: 2.0e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 10 seconds
Window 6 [1:173093312-213583529]
Reference markers: 1,151,258
Study markers: 1,960
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 15330
Estimated err: 6.6e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 7 [1:211080854-240239399]
Reference markers: 854,129
Study markers: 1,793
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Burnin iteration 3: 1 second
Estimated ne: 8274
Estimated err: 7.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 6 seconds
Window 8 [1:239251841-249240543]
Reference markers: 306,271
Study markers: 716
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 16635
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 1 second
Cumulative Statistics:
Reference markers: 6,468,094
Study markers: 12,391
Haplotype phasing time: 42 seconds
Imputation time: 1 minute 5 seconds
Total time: 3 minutes 49 seconds
End time: 04:51 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.1.vcf ref=ALL.chr1.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr1.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.1
2024-09-27 16:52:00.610 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:52:14.012 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 16:54:52.578 Hail: INFO: wrote matrix table with 6468094 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.1.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.2.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 2
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.2
63761 MB RAM detected; reserving 31880 MB for main workspace.
13813 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999898.
13813 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.2.bed +
SampleData1/Fold_0/test_data.clumped.pruned.2.bim +
SampleData1/Fold_0/test_data.clumped.pruned.2.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.2.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.2
--chr 2
--out SampleData1/Fold_0/hail.test.2
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
13813 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999898.
13813 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.2.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 04:54 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.2.vcf
ref=ALL.chr2.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr2.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.2
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [2:10179-18145025]
Reference markers: 573,772
Study markers: 1,369
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 7930
Estimated err: 2.6e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 2 [2:16417586-49114274]
Reference markers: 1,053,726
Study markers: 1,900
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16394
Estimated err: 8.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 3 [2:47323299-88186309]
Reference markers: 1,255,428
Study markers: 1,915
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16727
Estimated err: 6.6e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 4 [2:86643697-134624574]
Reference markers: 1,221,295
Study markers: 1,730
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 12237
Estimated err: 1.4e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 11 seconds
Window 5 [2:133390491-177333395]
Reference markers: 1,235,397
Study markers: 2,049
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 11427
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 1 second
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 6 [2:175084768-220694589]
Reference markers: 1,278,696
Study markers: 2,058
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 11824
Estimated err: 1.3e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 7 [2:218829611-242575162]
Reference markers: 734,807
Study markers: 1,751
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 17532
Estimated err: 2.1e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 8 [2:241170836-243188367]
Reference markers: 65,148
Study markers: 108
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 164880
Estimated err: 4.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 0 seconds
Cumulative Statistics:
Reference markers: 7,081,600
Study markers: 12,251
Haplotype phasing time: 40 seconds
Imputation time: 53 seconds
Total time: 3 minutes 43 seconds
End time: 04:58 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.2.vcf ref=ALL.chr2.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr2.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.2
2024-09-27 16:58:36.213 Hail: INFO: scanning VCF for sortedness...
2024-09-27 16:58:51.021 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:01:45.909 Hail: INFO: wrote matrix table with 7081600 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.2.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.3.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 3
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.3
63761 MB RAM detected; reserving 31880 MB for main workspace.
11785 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
11785 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.3.bed +
SampleData1/Fold_0/test_data.clumped.pruned.3.bim +
SampleData1/Fold_0/test_data.clumped.pruned.3.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.3.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.3
--chr 3
--out SampleData1/Fold_0/hail.test.3
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
11785 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
11785 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.3.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:01 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.3.vcf
ref=ALL.chr3.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr3.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.3
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [3:60069-21643990]
Reference markers: 718,343
Study markers: 1,639
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9255
Estimated err: 2.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 2 [3:19789059-60278775]
Reference markers: 1,194,873
Study markers: 1,958
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 14946
Estimated err: 7.7e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 3 [3:59462914-105372608]
Reference markers: 1,325,897
Study markers: 1,972
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 13192
Estimated err: 1.4e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 1 second
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 4 [3:103398092-143445761]
Reference markers: 1,153,131
Study markers: 2,033
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 15223
Estimated err: 1.3e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 5 [3:141741447-182238285]
Reference markers: 1,155,993
Study markers: 2,010
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16136
Estimated err: 8.0e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 6 [3:179790776-197962381]
Reference markers: 545,816
Study markers: 1,364
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 9843
Estimated err: 2.4e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Cumulative Statistics:
Reference markers: 5,832,276
Study markers: 10,457
Haplotype phasing time: 35 seconds
Imputation time: 40 seconds
Total time: 3 minutes 33 seconds
End time: 05:05 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.3.vcf ref=ALL.chr3.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr3.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.3
2024-09-27 17:05:19.815 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:05:31.288 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:07:53.762 Hail: INFO: wrote matrix table with 5832276 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.3.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.4.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 4
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.4
63761 MB RAM detected; reserving 31880 MB for main workspace.
11041 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999902.
11041 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.4.bed +
SampleData1/Fold_0/test_data.clumped.pruned.4.bim +
SampleData1/Fold_0/test_data.clumped.pruned.4.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.4.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.4
--chr 4
--out SampleData1/Fold_0/hail.test.4
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
11041 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999902.
11041 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.4.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:07 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.4.vcf
ref=ALL.chr4.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr4.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.4
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [4:10005-22520039]
Reference markers: 786,334
Study markers: 1,498
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 12681
Estimated err: 2.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Window 2 [4:20544690-60086601]
Reference markers: 1,127,621
Study markers: 1,878
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Estimated ne: 14831
Estimated err: 7.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 3 [4:57715343-105764064]
Reference markers: 1,443,530
Study markers: 2,053
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Estimated ne: 14756
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Window 4 [4:102709973-150650017]
Reference markers: 1,376,146
Study markers: 2,059
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Estimated ne: 13702
Estimated err: 1.3e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Window 5 [4:148086441-182155049]
Reference markers: 1,012,842
Study markers: 1,896
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Estimated ne: 13437
Estimated err: 7.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 6 [4:181312927-191043881]
Reference markers: 316,589
Study markers: 921
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 9458
Estimated err: 8.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 2 seconds
Cumulative Statistics:
Reference markers: 5,732,585
Study markers: 9,762
Haplotype phasing time: 20 seconds
Imputation time: 24 seconds
Total time: 2 minutes 8 seconds
End time: 05:10 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.4.vcf ref=ALL.chr4.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr4.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.4
2024-09-27 17:10:02.238 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:10:13.123 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:12:31.502 Hail: INFO: wrote matrix table with 5732585 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.4.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.5.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 5
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.5
63761 MB RAM detected; reserving 31880 MB for main workspace.
10632 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999882.
10632 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.5.bed +
SampleData1/Fold_0/test_data.clumped.pruned.5.bim +
SampleData1/Fold_0/test_data.clumped.pruned.5.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.5.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.5
--chr 5
--out SampleData1/Fold_0/hail.test.5
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
10632 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999882.
10632 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.5.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:12 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.5.vcf
ref=ALL.chr5.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr5.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.5
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [5:10043-24839896]
Reference markers: 785,833
Study markers: 1,647
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16474
Estimated err: 2.4e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Window 2 [5:22775078-69306446]
Reference markers: 1,297,643
Study markers: 1,975
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Estimated ne: 15101
Estimated err: 1.5e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Window 3 [5:67534999-111982027]
Reference markers: 1,223,992
Study markers: 2,011
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Estimated ne: 13613
Estimated err: 6.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 4 [5:109533719-149269607]
Reference markers: 1,201,670
Study markers: 1,971
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 12773
Estimated err: 7.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 5 [5:148226432-174766607]
Reference markers: 787,384
Study markers: 1,791
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 14571
Estimated err: 2.1e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Window 6 [5:174167120-180904689]
Reference markers: 201,966
Study markers: 483
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 10030
Estimated err: 2.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 1 second
Cumulative Statistics:
Reference markers: 5,265,763
Study markers: 9,449
Haplotype phasing time: 18 seconds
Imputation time: 22 seconds
Total time: 1 minute 44 seconds
End time: 05:14 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.5.vcf ref=ALL.chr5.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr5.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.5
2024-09-27 17:14:15.812 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:14:25.845 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:16:34.038 Hail: INFO: wrote matrix table with 5265763 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.5.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.6.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 6
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.6
63761 MB RAM detected; reserving 31880 MB for main workspace.
10068 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999902.
10068 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.6.bed +
SampleData1/Fold_0/test_data.clumped.pruned.6.bim +
SampleData1/Fold_0/test_data.clumped.pruned.6.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.6.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.6
--chr 6
--out SampleData1/Fold_0/hail.test.6
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
10068 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999902.
10068 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.6.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:16 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.6.vcf
ref=ALL.chr6.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr6.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.6
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [6:63854-21294216]
Reference markers: 658,963
Study markers: 1,608
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 12514
Estimated err: 2.4e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 2 [6:20005714-56895330]
Reference markers: 1,115,840
Study markers: 1,951
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 12421
Estimated err: 1.3e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 1 second
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 3 [6:53556519-110284141]
Reference markers: 1,605,180
Study markers: 2,135
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 17115
Estimated err: 6.6e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 1 second
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 9 seconds
Window 4 [6:108297729-149276979]
Reference markers: 1,157,048
Study markers: 2,011
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 14130
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 1 second
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 5 [6:148481212-170715177]
Reference markers: 694,787
Study markers: 1,585
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 16045
Estimated err: 8.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 6 [6:169508794-171053406]
Reference markers: 46,342
Study markers: 84
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 168185
Estimated err: 7.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 0 seconds
Cumulative Statistics:
Reference markers: 5,024,119
Study markers: 8,898
Haplotype phasing time: 31 seconds
Imputation time: 36 seconds
Total time: 2 minutes 55 seconds
End time: 05:19 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.6.vcf ref=ALL.chr6.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr6.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.6
2024-09-27 17:19:29.657 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:19:39.682 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:21:41.817 Hail: INFO: wrote matrix table with 5024119 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.6.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.7.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 7
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.7
63761 MB RAM detected; reserving 31880 MB for main workspace.
9496 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999897.
9496 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.7.bed +
SampleData1/Fold_0/test_data.clumped.pruned.7.bim +
SampleData1/Fold_0/test_data.clumped.pruned.7.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.7.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.7
--chr 7
--out SampleData1/Fold_0/hail.test.7
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
9496 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999897.
9496 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.7.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:21 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.7.vcf
ref=ALL.chr7.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr7.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.7
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [7:14808-22507950]
Reference markers: 834,932
Study markers: 1,538
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 13893
Estimated err: 1.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 2 [7:21712814-53284120]
Reference markers: 941,559
Study markers: 1,883
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16574
Estimated err: 1.6e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 3 [7:51371519-102207161]
Reference markers: 1,373,374
Study markers: 1,942
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 12995
Estimated err: 1.1e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 9 seconds
Window 4 [7:98972040-141848611]
Reference markers: 1,207,792
Study markers: 2,091
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 14555
Estimated err: 6.7e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 5 [7:140215356-159128567]
Reference markers: 589,076
Study markers: 1,337
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 7130
Estimated err: 2.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Cumulative Statistics:
Reference markers: 4,716,715
Study markers: 8,420
Haplotype phasing time: 28 seconds
Imputation time: 35 seconds
Total time: 2 minutes 43 seconds
End time: 05:24 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.7.vcf ref=ALL.chr7.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr7.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.7
2024-09-27 17:24:25.990 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:24:35.374 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:26:29.775 Hail: INFO: wrote matrix table with 4716715 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.7.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.8.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 8
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.8
63761 MB RAM detected; reserving 31880 MB for main workspace.
8867 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8867 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.8.bed +
SampleData1/Fold_0/test_data.clumped.pruned.8.bim +
SampleData1/Fold_0/test_data.clumped.pruned.8.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.8.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.8
--chr 8
--out SampleData1/Fold_0/hail.test.8
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8867 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8867 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.8.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:26 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.8.vcf
ref=ALL.chr8.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr8.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.8
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [8:11740-18799855]
Reference markers: 877,773
Study markers: 1,470
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 0 seconds
Estimated ne: 8676
Estimated err: 9.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 9 seconds
Window 2 [8:17678413-59729351]
Reference markers: 1,232,933
Study markers: 1,987
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 15278
Estimated err: 7.0e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
java.lang.IllegalArgumentException: Duplicate marker: 8 89329148 . G A
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 17:28:22.413 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.8.vcf ref=ALL.chr8.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr8.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.8
2024-09-27 17:28:26.744 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:29:17.542 Hail: INFO: wrote matrix table with 2039669 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.8.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.9.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 9
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.9
63761 MB RAM detected; reserving 31880 MB for main workspace.
7768 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999892.
7768 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.9.bed +
SampleData1/Fold_0/test_data.clumped.pruned.9.bim +
SampleData1/Fold_0/test_data.clumped.pruned.9.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.9.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.9
--chr 9
--out SampleData1/Fold_0/hail.test.9
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
7768 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999892.
7768 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.9.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:29 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.9.vcf
ref=ALL.chr9.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr9.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.9
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [9:10163-19603557]
Reference markers: 766,446
Study markers: 1,563
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 8930
Estimated err: 2.0e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 2 [9:18636347-80220415]
Reference markers: 1,009,966
Study markers: 1,725
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 12947
Estimated err: 8.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 3 [9:79151500-112369214]
Reference markers: 978,438
Study markers: 1,894
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 12438
Estimated err: 7.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 4 [9:110879571-137041122]
Reference markers: 775,425
Study markers: 1,625
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 15593
Estimated err: 8.7e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 5 [9:136539237-141148854]
Reference markers: 166,298
Study markers: 361
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 12562
Estimated err: 7.8e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Cumulative Statistics:
Reference markers: 3,560,687
Study markers: 6,879
Haplotype phasing time: 24 seconds
Imputation time: 31 seconds
Total time: 2 minutes 2 seconds
End time: 05:31 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.9.vcf ref=ALL.chr9.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr9.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.9
2024-09-27 17:31:20.202 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:31:27.445 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:32:53.929 Hail: INFO: wrote matrix table with 3560687 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.9.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.10.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 10
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.10
63761 MB RAM detected; reserving 31880 MB for main workspace.
8824 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
8824 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.10.bed +
SampleData1/Fold_0/test_data.clumped.pruned.10.bim +
SampleData1/Fold_0/test_data.clumped.pruned.10.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.10.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.10
--chr 10
--out SampleData1/Fold_0/hail.test.10
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8824 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
8824 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.10.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:32 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.10.vcf
ref=ALL.chr10.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr10.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.10
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [10:60494-18538847]
Reference markers: 622,290
Study markers: 1,540
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9145
Estimated err: 1.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 2 [10:17068251-54798179]
Reference markers: 1,000,400
Study markers: 1,557
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 11245
Estimated err: 1.6e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 6 seconds
Window 3 [10:53792357-90734239]
Reference markers: 1,092,804
Study markers: 1,949
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 13861
Estimated err: 1.4e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 4 [10:88784362-123900556]
Reference markers: 1,013,192
Study markers: 1,914
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16709
Estimated err: 7.7e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 1 second
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 6 seconds
Window 5 [10:123093921-135524373]
Reference markers: 406,013
Study markers: 1,114
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 11500
Estimated err: 2.8e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 2 seconds
Cumulative Statistics:
Reference markers: 3,992,219
Study markers: 7,769
Haplotype phasing time: 26 seconds
Imputation time: 30 seconds
Total time: 2 minutes 14 seconds
End time: 05:35 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.10.vcf ref=ALL.chr10.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr10.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.10
2024-09-27 17:35:08.632 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:35:16.600 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:36:57.724 Hail: INFO: wrote matrix table with 3992219 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.10.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.11.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 11
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.11
63761 MB RAM detected; reserving 31880 MB for main workspace.
8420 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.9999.
8420 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.11.bed +
SampleData1/Fold_0/test_data.clumped.pruned.11.bim +
SampleData1/Fold_0/test_data.clumped.pruned.11.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.11.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.11
--chr 11
--out SampleData1/Fold_0/hail.test.11
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8420 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.9999.
8420 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.11.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:36 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.11.vcf
ref=ALL.chr11.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr11.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.11
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [11:61395-22535739]
Reference markers: 711,691
Study markers: 1,628
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 17448
Estimated err: 2.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 2 [11:21349458-70803324]
Reference markers: 1,428,012
Study markers: 2,039
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 15187
Estimated err: 1.4e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 10 seconds
Window 3 [11:69604836-112290497]
Reference markers: 1,283,964
Study markers: 2,121
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16129
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 4 [11:110168424-133043804]
Reference markers: 692,681
Study markers: 1,732
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 19162
Estimated err: 2.1e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Window 5 [11:132350788-134946453]
Reference markers: 88,218
Study markers: 272
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 20805
Estimated err: 1.2e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 0 seconds
Cumulative Statistics:
Reference markers: 4,045,628
Study markers: 7,450
Haplotype phasing time: 25 seconds
Imputation time: 29 seconds
Total time: 2 minutes 31 seconds
End time: 05:39 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.11.vcf ref=ALL.chr11.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr11.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.11
2024-09-27 17:39:29.012 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:39:37.312 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:41:17.495 Hail: INFO: wrote matrix table with 4045628 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.11.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.12.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 12
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.12
63761 MB RAM detected; reserving 31880 MB for main workspace.
8198 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8198 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.12.bed +
SampleData1/Fold_0/test_data.clumped.pruned.12.bim +
SampleData1/Fold_0/test_data.clumped.pruned.12.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.12.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.12
--chr 12
--out SampleData1/Fold_0/hail.test.12
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
8198 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999891.
8198 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.12.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:41 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.12.vcf
ref=ALL.chr12.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr12.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.12
nthreads=8
java.lang.IllegalArgumentException: Duplicate marker: 12 8400000 . T G
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 17:41:36.003 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.12.vcf ref=ALL.chr12.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr12.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.12
2024-09-27 17:41:36.194 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 17:41:36.688 Hail: INFO: wrote matrix table with 0 rows and 95 columns in 0 partitions to SampleData1/Fold_0/hail.test.12.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.13.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 13
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.13
63761 MB RAM detected; reserving 31880 MB for main workspace.
6350 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
6350 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.13.bed +
SampleData1/Fold_0/test_data.clumped.pruned.13.bim +
SampleData1/Fold_0/test_data.clumped.pruned.13.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.13.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.13
--chr 13
--out SampleData1/Fold_0/hail.test.13
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
6350 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999901.
6350 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.13.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:41 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.13.vcf
ref=ALL.chr13.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr13.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.13
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [13:19020047-40892639]
Reference markers: 671,822
Study markers: 1,479
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 13287
Estimated err: 2.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 5 seconds
Window 2 [13:39849087-81726083]
Reference markers: 1,198,543
Study markers: 1,989
Burnin iteration 1: 1 second
Burnin iteration 2: 0 seconds
Estimated ne: 13447
Estimated err: 1.3e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 3 [13:79491658-110702359]
Reference markers: 937,844
Study markers: 1,859
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 18191
Estimated err: 7.0e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Window 4 [13:109580155-115109852]
Reference markers: 180,096
Study markers: 464
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 11961
Estimated err: 1.4e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 1 second
Cumulative Statistics:
Reference markers: 2,857,916
Study markers: 5,503
Haplotype phasing time: 12 seconds
Imputation time: 13 seconds
Total time: 1 minute 10 seconds
End time: 05:42 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.13.vcf ref=ALL.chr13.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr13.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.13
2024-09-27 17:42:47.366 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:42:52.923 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:44:02.178 Hail: INFO: wrote matrix table with 2857916 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.13.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.14.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 14
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.14
63761 MB RAM detected; reserving 31880 MB for main workspace.
5742 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999852.
5742 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.14.bed +
SampleData1/Fold_0/test_data.clumped.pruned.14.bim +
SampleData1/Fold_0/test_data.clumped.pruned.14.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.14.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.14
--chr 14
--out SampleData1/Fold_0/hail.test.14
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5742 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999852.
5742 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.14.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:44 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.14.vcf
ref=ALL.chr14.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr14.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.14
nthreads=8
java.lang.IllegalArgumentException: Duplicate marker: 14 21649957 . C T
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 17:44:22.736 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.14.vcf ref=ALL.chr14.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr14.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.14
2024-09-27 17:44:22.955 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 17:44:23.429 Hail: INFO: wrote matrix table with 0 rows and 95 columns in 0 partitions to SampleData1/Fold_0/hail.test.14.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.15.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 15
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.15
63761 MB RAM detected; reserving 31880 MB for main workspace.
5569 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
5569 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.15.bed +
SampleData1/Fold_0/test_data.clumped.pruned.15.bim +
SampleData1/Fold_0/test_data.clumped.pruned.15.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.15.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.15
--chr 15
--out SampleData1/Fold_0/hail.test.15
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5569 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
5569 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.15.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:44 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.15.vcf
ref=ALL.chr15.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr15.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.15
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [15:20000041-32890838]
Reference markers: 325,223
Study markers: 632
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 8658
Estimated err: 5.1e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 50 seconds
Window 2 [15:32564155-62934668]
Reference markers: 905,216
Study markers: 1,640
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 7920
Estimated err: 2.1e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 6 seconds
Window 3 [15:61592413-93656456]
Reference markers: 943,933
Study markers: 1,788
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 13708
Estimated err: 7.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 4 seconds
Window 4 [15:92986651-102521131]
Reference markers: 318,635
Study markers: 973
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 7332
Estimated err: 1.1e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 2 seconds
Cumulative Statistics:
Reference markers: 2,424,689
Study markers: 4,873
Haplotype phasing time: 17 seconds
Imputation time: 1 minute 3 seconds
Total time: 1 minute 32 seconds
End time: 05:45 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.15.vcf ref=ALL.chr15.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr15.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.15
2024-09-27 17:45:55.571 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:46:00.603 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:47:01.284 Hail: INFO: wrote matrix table with 2424689 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.15.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.16.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 16
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.16
63761 MB RAM detected; reserving 31880 MB for main workspace.
6069 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99988.
6069 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.16.bed +
SampleData1/Fold_0/test_data.clumped.pruned.16.bim +
SampleData1/Fold_0/test_data.clumped.pruned.16.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.16.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.16
--chr 16
--out SampleData1/Fold_0/hail.test.16
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
6069 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99988.
6069 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.16.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:47 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.16.vcf
ref=ALL.chr16.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr16.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.16
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [16:60086-19243846]
Reference markers: 737,912
Study markers: 1,489
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 7912
Estimated err: 2.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 10 seconds
Window 2 [16:18062715-58867225]
Reference markers: 828,530
Study markers: 1,634
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 0 seconds
Estimated ne: 9672
Estimated err: 2.0e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 1 second
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 6 seconds
java.util.zip.ZipException: invalid distance too far back
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:165)
at java.base/java.util.zip.GZIPInputStream.read(GZIPInputStream.java:118)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
at blbutil.BGZipIt.inflateBlock(BGZipIt.java:314)
at blbutil.BGZipIt.lambda$readAndInflateBlocks$3(BGZipIt.java:263)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.Nodes$SizedCollectorTask.compute(Nodes.java:1886)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.helpCC(ForkJoinPool.java:1115)
at java.base/java.util.concurrent.ForkJoinPool.externalHelpComplete(ForkJoinPool.java:1957)
at java.base/java.util.concurrent.ForkJoinTask.tryExternalHelp(ForkJoinTask.java:378)
at java.base/java.util.concurrent.ForkJoinTask.externalAwaitDone(ForkJoinTask.java:323)
at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:412)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736)
at java.base/java.util.stream.Nodes.collect(Nodes.java:333)
at java.base/java.util.stream.ReferencePipeline.evaluateToNode(ReferencePipeline.java:109)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
at blbutil.BGZipIt.readAndInflateBlocks(BGZipIt.java:264)
at blbutil.BGZipIt.fillBuffer(BGZipIt.java:148)
at blbutil.BGZipIt.next(BGZipIt.java:137)
at blbutil.BGZipIt.next(BGZipIt.java:53)
at blbutil.BlockLineReader.lambda$startFileReadingThread$0(BlockLineReader.java:107)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 17:48:18.737 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.16.vcf ref=ALL.chr16.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr16.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.16
2024-09-27 17:48:22.556 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:49:01.126 Hail: INFO: wrote matrix table with 1522700 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.16.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.17.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 17
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.17
63761 MB RAM detected; reserving 31880 MB for main workspace.
5723 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99% done.
Total genotyping rate is 0.999884.
5723 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.17.bed +
SampleData1/Fold_0/test_data.clumped.pruned.17.bim +
SampleData1/Fold_0/test_data.clumped.pruned.17.fam ... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.17.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.17
--chr 17
--out SampleData1/Fold_0/hail.test.17
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5723 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99% done.
Total genotyping rate is 0.999884.
5723 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.17.vcf ... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:49 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.17.vcf
ref=ALL.chr17.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr17.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.17
nthreads=8
java.lang.IllegalArgumentException: Duplicate marker: 17 1144632 . C CT
at vcf.Markers.markerSet(Markers.java:129)
at vcf.Markers.<init>(Markers.java:83)
at vcf.Markers.create(Markers.java:62)
at vcf.RefGT.<init>(RefGT.java:90)
at vcf.RefTargSlidingWindow$Reader.window(RefTargSlidingWindow.java:326)
at vcf.RefTargSlidingWindow$Reader.readWindow(RefTargSlidingWindow.java:303)
at vcf.RefTargSlidingWindow$Reader.run(RefTargSlidingWindow.java:246)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 17:49:17.082 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.17.vcf ref=ALL.chr17.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr17.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.17
2024-09-27 17:49:17.277 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 17:49:17.685 Hail: INFO: wrote matrix table with 0 rows and 95 columns in 0 partitions to SampleData1/Fold_0/hail.test.17.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.18.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 18
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.18
63761 MB RAM detected; reserving 31880 MB for main workspace.
5578 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
5578 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.18.bed +
SampleData1/Fold_0/test_data.clumped.pruned.18.bim +
SampleData1/Fold_0/test_data.clumped.pruned.18.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.18.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.18
--chr 18
--out SampleData1/Fold_0/hail.test.18
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
5578 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
5578 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.18.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:49 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.18.vcf
ref=ALL.chr18.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr18.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.18
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [18:10083-15185474]
Reference markers: 478,762
Study markers: 1,290
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9011
Estimated err: 2.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 2 [18:13460439-56841603]
Reference markers: 1,173,645
Study markers: 2,084
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16300
Estimated err: 7.5e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 1 second
Phasing iteration 5: 1 second
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 8 seconds
Window 3 [18:55786342-76632429]
Reference markers: 654,683
Study markers: 1,588
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16706
Estimated err: 8.8e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Window 4 [18:75886666-78017156]
Reference markers: 77,128
Study markers: 182
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 84265
Estimated err: 5.4e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 0 seconds
Cumulative Statistics:
Reference markers: 2,267,185
Study markers: 4,917
Haplotype phasing time: 17 seconds
Imputation time: 18 seconds
Total time: 1 minute 24 seconds
End time: 05:50 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.18.vcf ref=ALL.chr18.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr18.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.18
2024-09-27 17:50:41.984 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:50:46.764 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:51:43.016 Hail: INFO: wrote matrix table with 2267185 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.18.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.19.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 19
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.19
63761 MB RAM detected; reserving 31880 MB for main workspace.
4364 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99987.
4364 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.19.bed +
SampleData1/Fold_0/test_data.clumped.pruned.19.bim +
SampleData1/Fold_0/test_data.clumped.pruned.19.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.19.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.19
--chr 19
--out SampleData1/Fold_0/hail.test.19
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
4364 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.99987.
4364 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.19.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:51 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.19.vcf
ref=ALL.chr19.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr19.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.19
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [19:60842-15654475]
Reference markers: 522,636
Study markers: 1,251
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Estimated ne: 12625
Estimated err: 2.3e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 6 seconds
Window 2 [19:14657353-48467440]
Reference markers: 980,357
Study markers: 1,641
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 16421
Estimated err: 2.6e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 3 [19:47339481-59118924]
Reference markers: 395,275
Study markers: 1,022
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 8805
Estimated err: 2.5e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Cumulative Statistics:
Reference markers: 1,832,506
Study markers: 3,773
Haplotype phasing time: 13 seconds
Imputation time: 16 seconds
Total time: 1 minute 11 seconds
End time: 05:52 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.19.vcf ref=ALL.chr19.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr19.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.19
2024-09-27 17:52:54.163 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:52:58.073 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:53:45.037 Hail: INFO: wrote matrix table with 1832506 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.19.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.20.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 20
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.20
63761 MB RAM detected; reserving 31880 MB for main workspace.
4916 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
4916 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.20.bed +
SampleData1/Fold_0/test_data.clumped.pruned.20.bim +
SampleData1/Fold_0/test_data.clumped.pruned.20.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.20.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.20
--chr 20
--out SampleData1/Fold_0/hail.test.20
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
4916 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999887.
4916 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.20.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:53 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.20.vcf
ref=ALL.chr20.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr20.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.20
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [20:60343-17038340]
Reference markers: 507,580
Study markers: 1,470
Burnin iteration 1: 2 seconds
Burnin iteration 2: 1 second
Burnin iteration 3: 1 second
Estimated ne: 9206
Estimated err: 1.9e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 2 [20:16012786-51450735]
Reference markers: 956,124
Study markers: 1,807
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Estimated ne: 15582
Estimated err: 7.2e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 7 seconds
Window 3 [20:50160268-62965354]
Reference markers: 419,983
Study markers: 1,208
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Estimated ne: 15124
Estimated err: 2.4e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 2 seconds
Cumulative Statistics:
Reference markers: 1,812,841
Study markers: 4,333
Haplotype phasing time: 15 seconds
Imputation time: 16 seconds
Total time: 1 minute 8 seconds
End time: 05:54 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.20.vcf ref=ALL.chr20.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr20.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.20
2024-09-27 17:54:53.622 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:54:57.517 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:55:43.685 Hail: INFO: wrote matrix table with 1812841 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.20.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.21.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 21
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.21
63761 MB RAM detected; reserving 31880 MB for main workspace.
2811 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
2811 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.21.bed +
SampleData1/Fold_0/test_data.clumped.pruned.21.bim +
SampleData1/Fold_0/test_data.clumped.pruned.21.fam ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.21.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.21
--chr 21
--out SampleData1/Fold_0/hail.test.21
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
2811 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989 done.
Total genotyping rate is 0.999906.
2811 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.21.vcf ... 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:55 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.21.vcf
ref=ALL.chr21.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr21.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.21
nthreads=8
Reference samples: 2,504
Study samples: 95
Window 1 [21:9411239-36416332]
Reference markers: 719,753
Study markers: 1,463
Burnin iteration 1: 1 second
Burnin iteration 2: 1 second
Burnin iteration 3: 0 seconds
Estimated ne: 11785
Estimated err: 2.4e-04
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 9 seconds
Window 2 [21:35704550-48119740]
Reference markers: 407,769
Study markers: 1,059
Burnin iteration 1: 0 seconds
Burnin iteration 2: 0 seconds
Burnin iteration 3: 0 seconds
Estimated ne: 8340
Estimated err: 1.1e-03
Phasing iteration 1: 0 seconds
Phasing iteration 2: 0 seconds
Phasing iteration 3: 0 seconds
Phasing iteration 4: 0 seconds
Phasing iteration 5: 0 seconds
Phasing iteration 6: 0 seconds
Phasing iteration 7: 0 seconds
Phasing iteration 8: 0 seconds
Phasing iteration 9: 0 seconds
Phasing iteration 10: 0 seconds
Phasing iteration 11: 0 seconds
Phasing iteration 12: 0 seconds
Imputation time: 3 seconds
Cumulative Statistics:
Reference markers: 1,105,538
Study markers: 2,450
Haplotype phasing time: 8 seconds
Imputation time: 12 seconds
Total time: 43 seconds
End time: 05:56 PM AEST on 27 Sep 2024
beagle.06Aug24.a91.jar finished
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.21.vcf ref=ALL.chr21.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr21.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.21
2024-09-27 17:56:26.726 Hail: INFO: scanning VCF for sortedness...
2024-09-27 17:56:29.161 Hail: INFO: Coerced prefix-sorted VCF, requiring additional sorting within data partitions on each query.
2024-09-27 17:56:56.949 Hail: INFO: wrote matrix table with 1105538 rows and 95 columns in 1 partition to SampleData1/Fold_0/hail.test.21.mt
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/test_data.clumped.pruned.22.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned
--chr 22
--make-bed
--out SampleData1/Fold_0/test_data.clumped.pruned.22
63761 MB RAM detected; reserving 31880 MB for main workspace.
2831 out of 172878 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99% done.
Total genotyping rate is 0.999896.
2831 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--make-bed to SampleData1/Fold_0/test_data.clumped.pruned.22.bed +
SampleData1/Fold_0/test_data.clumped.pruned.22.bim +
SampleData1/Fold_0/test_data.clumped.pruned.22.fam ... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%done.
PLINK v1.90b7.2 64-bit (11 Dec 2023) www.cog-genomics.org/plink/1.9/
(C) 2005-2023 Shaun Purcell, Christopher Chang GNU General Public License v3
Logging to SampleData1/Fold_0/hail.test.22.log.
Options in effect:
--bfile SampleData1/Fold_0/test_data.clumped.pruned.22
--chr 22
--out SampleData1/Fold_0/hail.test.22
--recode vcf
63761 MB RAM detected; reserving 31880 MB for main workspace.
2831 variants loaded from .bim file.
95 people (44 males, 51 females) loaded from .fam.
95 phenotype values loaded from .fam.
Using 1 thread (no multithreaded calculations invoked).
Before main variant filters, 95 founders and 0 nonfounders present.
Calculating allele frequencies... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99% done.
Total genotyping rate is 0.999896.
2831 variants and 95 people pass filters and QC.
Phenotype data is quantitative.
--recode vcf to SampleData1/Fold_0/hail.test.22.vcf ... 0%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%done.
beagle.06Aug24.a91.jar (version 5.4)
Copyright (C) 2014-2022 Brian L. Browning
Enter "java -jar beagle.06Aug24.a91.jar" to list command line argument
Start time: 05:56 PM AEST on 27 Sep 2024
Command line: java -Xmx51200m -jar beagle.06Aug24.a91.jar
gt=SampleData1/Fold_0/hail.test.22.vcf
ref=ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz
map=plink.chr22.GRCh37.map
out=SampleData1/Fold_0/beagle.hail.test.22
nthreads=8
java.util.zip.ZipException: invalid distance too far back
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:165)
at java.base/java.util.zip.GZIPInputStream.read(GZIPInputStream.java:118)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
at blbutil.BGZipIt.inflateBlock(BGZipIt.java:314)
at blbutil.BGZipIt.lambda$readAndInflateBlocks$3(BGZipIt.java:263)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.Nodes$SizedCollectorTask.compute(Nodes.java:1886)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736)
at java.base/java.util.stream.Nodes.collect(Nodes.java:333)
at java.base/java.util.stream.ReferencePipeline.evaluateToNode(ReferencePipeline.java:109)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
at blbutil.BGZipIt.readAndInflateBlocks(BGZipIt.java:264)
at blbutil.BGZipIt.fillBuffer(BGZipIt.java:148)
at blbutil.BGZipIt.next(BGZipIt.java:137)
at blbutil.BGZipIt.next(BGZipIt.java:53)
at blbutil.BlockLineReader.lambda$startFileReadingThread$0(BlockLineReader.java:107)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Terminating program.
2024-09-27 17:57:11.246 Hail: INFO: scanning VCF for sortedness...
java -Xmx50g -jar beagle gt=SampleData1/Fold_0/hail.test.22.vcf ref=ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz map=plink.chr22.GRCh37.map out=SampleData1/Fold_0/beagle.hail.test.22
2024-09-27 17:57:11.446 Hail: INFO: Coerced sorted VCF - no additional import work to do
2024-09-27 17:57:12.056 Hail: INFO: wrote matrix table with 0 rows and 95 columns in 0 partitions to SampleData1/Fold_0/hail.test.22.mt
showing top 5 rows
None
1 SampleData1/Fold_0/hail.test.1.mt (6468094, 95) (6468094, 95)
2 SampleData1/Fold_0/hail.test.2.mt (7081600, 95) (13549694, 95)
3 SampleData1/Fold_0/hail.test.3.mt (5832276, 95) (19381970, 95)
4 SampleData1/Fold_0/hail.test.4.mt (5732585, 95) (25114555, 95)
5 SampleData1/Fold_0/hail.test.5.mt (5265763, 95) (30380318, 95)
6 SampleData1/Fold_0/hail.test.6.mt (5024119, 95) (35404437, 95)
7 SampleData1/Fold_0/hail.test.7.mt (4716715, 95) (40121152, 95)
8 SampleData1/Fold_0/hail.test.8.mt (2039669, 95) (42160821, 95)
9 SampleData1/Fold_0/hail.test.9.mt (3560687, 95) (45721508, 95)
10 SampleData1/Fold_0/hail.test.10.mt (3992219, 95) (49713727, 95)
11 SampleData1/Fold_0/hail.test.11.mt (4045628, 95) (53759355, 95)
12 SampleData1/Fold_0/hail.test.12.mt (0, 95) (53759355, 95)
13 SampleData1/Fold_0/hail.test.13.mt (2857916, 95) (56617271, 95)
14 SampleData1/Fold_0/hail.test.14.mt (0, 95) (56617271, 95)
15 SampleData1/Fold_0/hail.test.15.mt (2424689, 95) (59041960, 95)
16 SampleData1/Fold_0/hail.test.16.mt (1522700, 95) (60564660, 95)
17 SampleData1/Fold_0/hail.test.17.mt (0, 95) (60564660, 95)
18 SampleData1/Fold_0/hail.test.18.mt (2267185, 95) (62831845, 95)
19 SampleData1/Fold_0/hail.test.19.mt (1832506, 95) (64664351, 95)
20 SampleData1/Fold_0/hail.test.20.mt (1812841, 95) (66477192, 95)
21 SampleData1/Fold_0/hail.test.21.mt (1105538, 95) (67582730, 95)
22 SampleData1/Fold_0/hail.test.22.mt (0, 95) (67582730, 95)
(67582730, 95)
2024-09-27 17:59:42.287 Hail: INFO: wrote matrix table with 67582730 rows and 95 columns in 18 partitions to SampleData1/Fold_0/HAILTEST.mt
showing top 5 rows
None
2024-09-27 17:59:43.085 Hail: INFO: Reading table to impute column types
2024-09-27 17:59:44.498 Hail: INFO: Finished type imputation (0 + 1) / 1]
Loading field 'chr_name' as type int32 (imputed)
Loading field 'chr_position' as type int32 (imputed)
Loading field 'effect_allele' as type str (imputed)
Loading field 'other_allele' as type str (imputed)
Loading field 'effect_weight' as type float64 (imputed)
2024-09-27 18:00:29.825 Hail: INFO: Ordering unsorted dataset with network shuffle
2024-09-27 18:00:31.442 Hail: INFO: Ordering unsorted dataset with network shuffle
2024-09-27 18:04:23.664 Hail: INFO: Ordering unsorted dataset with network shuffle
2024-09-27 18:14:10.295 Hail: INFO: Coerced sorted dataset====> (17 + 1) / 18]
2024-09-27 18:14:10.414 Hail: INFO: merging 9 files totalling 2.5K...
2024-09-27 18:14:10.424 Hail: INFO: while writing:
SampleData1/Fold_0/HAIL/test_PRS.txt
merge time: 9.174ms
2024-09-27 18:14:10.864 Hail: INFO: Reading table to impute column types
2024-09-27 18:14:11.204 Hail: INFO: Finished type imputation
Loading field 's' as type str (imputed)
Loading field 'prs' as type float64 (imputed)
2024-09-27 18:14:11.486 Hail: INFO: Coerced sorted dataset
subjectID PGS002724
0 HG00096_HG00096 -40.344
1 HG00109_HG00109 -23.098
2 HG00112_HG00112 2.3007
3 HG00119_HG00119 -4.378
4 HG00125_HG00125 9.9591
subjectID PGS002724
0 HG00096_HG00096 -40.3440
1 HG00109_HG00109 -23.0980
2 HG00112_HG00112 2.3007
3 HG00119_HG00119 -4.3780
4 HG00125_HG00125 9.9591
Continous Phenotype!
subjectID PGS002724
0 HG00097_HG00097 18.05800
1 HG00099_HG00099 -35.68400
2 HG00101_HG00101 23.94300
3 HG00102_HG00102 -20.05600
4 HG00103_HG00103 17.40100
.. ... ...
375 NA20818_NA20818 6.77010
376 NA20826_NA20826 5.06360
377 NA20827_NA20827 9.35300
378 NA20828_NA20828 22.78500
379 NA20832_NA20832 0.96845
[380 rows x 2 columns]
SCORE FID IID
0 -40.3440 HG00096 HG00096
1 -23.0980 HG00109 HG00109
2 2.3007 HG00112 HG00112
3 -4.3780 HG00119 HG00119
4 9.9591 HG00125 HG00125
.. ... ... ...
90 17.7130 NA20814 NA20814
91 1.5582 NA20815 NA20815
92 19.4650 NA20819 NA20819
93 4.6435 NA20821 NA20821
94 17.9070 NA20822 NA20822
[95 rows x 3 columns]
FID IID Sex PC1 PC2 PC3 PC4 PC5 \
0 HG00097 HG00097 2 -0.001453 0.084820 0.006792 0.013653 0.027149
1 HG00099 HG00099 2 -0.002017 0.089514 -0.022355 0.001888 -0.000037
2 HG00101 HG00101 1 -0.000380 0.096056 -0.018231 -0.016026 0.012093
3 HG00102 HG00102 2 0.000292 0.071832 0.018087 -0.045180 0.028123
4 HG00103 HG00103 1 -0.008372 0.065005 -0.009089 -0.026468 -0.009184
.. ... ... ... ... ... ... ... ...
375 NA20818 NA20818 2 -0.047156 -0.040644 -0.052693 0.021050 -0.013389
376 NA20826 NA20826 2 -0.042629 -0.059404 -0.066130 0.006495 -0.009525
377 NA20827 NA20827 1 -0.044060 -0.053125 -0.065463 0.015030 -0.004314
378 NA20828 NA20828 2 -0.047621 -0.050577 -0.043164 0.003004 -0.016823
379 NA20832 NA20832 2 -0.041535 -0.049826 -0.047877 0.005951 -0.003770
PC6 SCORE
0 0.032581 18.05800
1 0.009107 -35.68400
2 0.019296 23.94300
3 -0.003620 -20.05600
4 -0.030565 17.40100
.. ... ...
375 -0.047403 6.77010
376 0.010779 5.06360
377 0.003873 9.35300
378 0.015832 22.78500
379 -0.023086 0.96845
[380 rows x 10 columns]
clump_p1 clump_r2 clump_kb p_window_size p_slide_size p_LD_threshold \
0 1 0.1 200 200 50 0.25
numberofpca tempalpha l1weight Train_pure_prs Train_null_model \
0 6 0.1 0.1 -399.690143 0.227477
Train_best_model Test_pure_prs Test_null_model Test_best_model
0 0.243729 -447.487354 0.14297 0.196619
Repeat the process for each fold.#
Change the foldnumber variable.
#foldnumber = sys.argv[1]
foldnumber = "0" # Setting 'foldnumber' to "0"
Or uncomment the following line:
# foldnumber = sys.argv[1]
python HAIL.py 0
python HAIL.py 1
python HAIL.py 2
python HAIL.py 3
python HAIL.py 4
The following files should exist after the execution:
SampleData1/Fold_0/HAIL/Results.csvSampleData1/Fold_1/HAIL/Results.csvSampleData1/Fold_2/HAIL/Results.csvSampleData1/Fold_3/HAIL/Results.csvSampleData1/Fold_4/HAIL/Results.csv
Check the results file for each fold.#
import os
# List of file names to check for existence
f = [
"./"+filedirec+"/Fold_0"+os.sep+result_directory+"Results.csv",
"./"+filedirec+"/Fold_1"+os.sep+result_directory+"Results.csv",
"./"+filedirec+"/Fold_2"+os.sep+result_directory+"Results.csv",
"./"+filedirec+"/Fold_3"+os.sep+result_directory+"Results.csv",
"./"+filedirec+"/Fold_4"+os.sep+result_directory+"Results.csv",
]
# Loop through each file name in the list
for loop in range(0,5):
# Check if the file exists in the specified directory for the given fold
if os.path.exists(filedirec+os.sep+"Fold_"+str(loop)+os.sep+result_directory+os.sep+"Results.csv"):
temp = pd.read_csv(filedirec+os.sep+"Fold_"+str(loop)+os.sep+result_directory+os.sep+"Results.csv")
print("Fold_",loop, "Yes, the file exists.")
#print(temp.head())
print("Number of P-values processed: ",len(temp))
# Print a message indicating that the file exists
else:
# Print a message indicating that the file does not exist
print("Fold_",loop, "No, the file does not exist.")
Fold_ 0 Yes, the file exists.
Number of P-values processed: 1
Fold_ 1 Yes, the file exists.
Number of P-values processed: 1
Fold_ 2 Yes, the file exists.
Number of P-values processed: 1
Fold_ 3 Yes, the file exists.
Number of P-values processed: 1
Fold_ 4 Yes, the file exists.
Number of P-values processed: 1
Sum the results for each fold.#
print("We have to ensure when we sum the entries across all Folds, the same rows are merged!")
def sum_and_average_columns(data_frames):
"""Sum and average numerical columns across multiple DataFrames, and keep non-numerical columns unchanged."""
# Initialize DataFrame to store the summed results for numerical columns
summed_df = pd.DataFrame()
non_numerical_df = pd.DataFrame()
for df in data_frames:
# Identify numerical and non-numerical columns
numerical_cols = df.select_dtypes(include=[np.number]).columns
non_numerical_cols = df.select_dtypes(exclude=[np.number]).columns
# Sum numerical columns
if summed_df.empty:
summed_df = pd.DataFrame(0, index=range(len(df)), columns=numerical_cols)
summed_df[numerical_cols] = summed_df[numerical_cols].add(df[numerical_cols], fill_value=0)
# Keep non-numerical columns (take the first non-numerical entry for each column)
if non_numerical_df.empty:
non_numerical_df = df[non_numerical_cols]
else:
non_numerical_df[non_numerical_cols] = non_numerical_df[non_numerical_cols].combine_first(df[non_numerical_cols])
# Divide the summed values by the number of dataframes to get the average
averaged_df = summed_df / len(data_frames)
# Combine numerical and non-numerical DataFrames
result_df = pd.concat([averaged_df, non_numerical_df], axis=1)
return result_df
from functools import reduce
import os
import pandas as pd
from functools import reduce
def find_common_rows(allfoldsframe):
# Define the performance columns that need to be excluded
performance_columns = [
'Train_null_model', 'Train_pure_prs', 'Train_best_model',
'Test_pure_prs', 'Test_null_model', 'Test_best_model'
]
important_columns = [
'clump_p1',
'clump_r2',
'clump_kb',
'p_window_size',
'p_slide_size',
'p_LD_threshold',
'pvalue',
'referencepanel',
'PRSice-2_Model',
'effectsizes',
'gemmamodel',
'numberofpca',
'tempalpha',
'l1weight',
]
# Function to remove performance columns from a DataFrame
def drop_performance_columns(df):
return df.drop(columns=performance_columns, errors='ignore')
def get_important_columns(df ):
existing_columns = [col for col in important_columns if col in df.columns]
if existing_columns:
return df[existing_columns].copy()
else:
return pd.DataFrame()
# Drop performance columns from all DataFrames in the list
allfoldsframe_dropped = [drop_performance_columns(df) for df in allfoldsframe]
# Get the important columns.
allfoldsframe_dropped = [get_important_columns(df) for df in allfoldsframe_dropped]
# Iteratively find common rows and track unique and common rows
common_rows = allfoldsframe_dropped[0]
for i in range(1, len(allfoldsframe_dropped)):
# Get the next DataFrame
next_df = allfoldsframe_dropped[i]
# Count unique rows in the current DataFrame and the next DataFrame
unique_in_common = common_rows.shape[0]
unique_in_next = next_df.shape[0]
# Find common rows between the current common_rows and the next DataFrame
common_rows = pd.merge(common_rows, next_df, how='inner')
# Count the common rows after merging
common_count = common_rows.shape[0]
# Print the unique and common row counts
print(f"Iteration {i}:")
print(f"Unique rows in current common DataFrame: {unique_in_common}")
print(f"Unique rows in next DataFrame: {unique_in_next}")
print(f"Common rows after merge: {common_count}\n")
# Now that we have the common rows, extract these from the original DataFrames
extracted_common_rows_frames = []
for original_df in allfoldsframe:
# Merge the common rows with the original DataFrame, keeping only the rows that match the common rows
extracted_common_rows = pd.merge(common_rows, original_df, how='inner', on=common_rows.columns.tolist())
# Add the DataFrame with the extracted common rows to the list
extracted_common_rows_frames.append(extracted_common_rows)
# Print the number of rows in the common DataFrames
for i, df in enumerate(extracted_common_rows_frames):
print(f"DataFrame {i + 1} with extracted common rows has {df.shape[0]} rows.")
# Return the list of DataFrames with extracted common rows
return extracted_common_rows_frames
# Example usage (assuming allfoldsframe is populated as shown earlier):
allfoldsframe = []
# Loop through each file name in the list
for loop in range(0, 5):
# Check if the file exists in the specified directory for the given fold
file_path = os.path.join(filedirec, "Fold_" + str(loop), result_directory, "Results.csv")
if os.path.exists(file_path):
allfoldsframe.append(pd.read_csv(file_path))
# Print a message indicating that the file exists
print("Fold_", loop, "Yes, the file exists.")
else:
# Print a message indicating that the file does not exist
print("Fold_", loop, "No, the file does not exist.")
# Find the common rows across all folds and return the list of extracted common rows
extracted_common_rows_list = find_common_rows(allfoldsframe)
# Sum the values column-wise
# For string values, do not sum it the values are going to be the same for each fold.
# Only sum the numeric values.
divided_result = sum_and_average_columns(extracted_common_rows_list)
print(divided_result)
We have to ensure when we sum the entries across all Folds, the same rows are merged!
Fold_ 0 Yes, the file exists.
Fold_ 1 Yes, the file exists.
Fold_ 2 Yes, the file exists.
Fold_ 3 Yes, the file exists.
Fold_ 4 Yes, the file exists.
Iteration 1:
Unique rows in current common DataFrame: 1
Unique rows in next DataFrame: 1
Common rows after merge: 1
Iteration 2:
Unique rows in current common DataFrame: 1
Unique rows in next DataFrame: 1
Common rows after merge: 1
Iteration 3:
Unique rows in current common DataFrame: 1
Unique rows in next DataFrame: 1
Common rows after merge: 1
Iteration 4:
Unique rows in current common DataFrame: 1
Unique rows in next DataFrame: 1
Common rows after merge: 1
DataFrame 1 with extracted common rows has 1 rows.
DataFrame 2 with extracted common rows has 1 rows.
DataFrame 3 with extracted common rows has 1 rows.
DataFrame 4 with extracted common rows has 1 rows.
DataFrame 5 with extracted common rows has 1 rows.
clump_p1 clump_r2 clump_kb p_window_size p_slide_size p_LD_threshold \
0 1.0 0.1 200.0 200.0 50.0 0.25
numberofpca tempalpha l1weight Train_pure_prs Train_null_model \
0 6.0 0.1 0.1 -408.190965 0.233856
Train_best_model Test_pure_prs Test_null_model Test_best_model
0 0.252479 -425.091606 0.137656 0.156992
Results#
1. Reporting Based on Best Training Performance:#
One can report the results based on the best performance of the training data. For example, if for a specific combination of hyperparameters, the training performance is high, report the corresponding test performance.
Example code:
df = divided_result.sort_values(by='Train_best_model', ascending=False) print(df.iloc[0].to_markdown())
Binary Phenotypes Result Analysis#
You can find the performance quality for binary phenotype using the following template:
This figure shows the 8 different scenarios that can exist in the results, and the following table explains each scenario.
We classified performance based on the following table:
Performance Level |
Range |
|---|---|
Low Performance |
0 to 0.5 |
Moderate Performance |
0.6 to 0.7 |
High Performance |
0.8 to 1 |
You can match the performance based on the following scenarios:
Scenario |
What’s Happening |
Implication |
|---|---|---|
High Test, High Train |
The model performs well on both training and test datasets, effectively learning the underlying patterns. |
The model is well-tuned, generalizes well, and makes accurate predictions on both datasets. |
High Test, Moderate Train |
The model generalizes well but may not be fully optimized on training data, missing some underlying patterns. |
The model is fairly robust but may benefit from further tuning or more training to improve its learning. |
High Test, Low Train |
An unusual scenario, potentially indicating data leakage or overestimation of test performance. |
The model’s performance is likely unreliable; investigate potential data issues or random noise. |
Moderate Test, High Train |
The model fits the training data well but doesn’t generalize as effectively, capturing only some test patterns. |
The model is slightly overfitting; adjustments may be needed to improve generalization on unseen data. |
Moderate Test, Moderate Train |
The model shows balanced but moderate performance on both datasets, capturing some patterns but missing others. |
The model is moderately fitting; further improvements could be made in both training and generalization. |
Moderate Test, Low Train |
The model underperforms on training data and doesn’t generalize well, leading to moderate test performance. |
The model may need more complexity, additional features, or better training to improve on both datasets. |
Low Test, High Train |
The model overfits the training data, performing poorly on the test set. |
The model doesn’t generalize well; simplifying the model or using regularization may help reduce overfitting. |
Low Test, Low Train |
The model performs poorly on both training and test datasets, failing to learn the data patterns effectively. |
The model is underfitting; it may need more complexity, additional features, or more data to improve performance. |
Recommendations for Publishing Results#
When publishing results, scenarios with moderate train and moderate test performance can be used for complex phenotypes or diseases. However, results showing high train and moderate test, high train and high test, and moderate train and high test are recommended.
For most phenotypes, results typically fall in the moderate train and moderate test performance category.
Continuous Phenotypes Result Analysis#
You can find the performance quality for continuous phenotypes using the following template:
This figure shows the 8 different scenarios that can exist in the results, and the following table explains each scenario.
We classified performance based on the following table:
Performance Level |
Range |
|---|---|
Low Performance |
0 to 0.2 |
Moderate Performance |
0.3 to 0.7 |
High Performance |
0.8 to 1 |
You can match the performance based on the following scenarios:
Scenario |
What’s Happening |
Implication |
|---|---|---|
High Test, High Train |
The model performs well on both training and test datasets, effectively learning the underlying patterns. |
The model is well-tuned, generalizes well, and makes accurate predictions on both datasets. |
High Test, Moderate Train |
The model generalizes well but may not be fully optimized on training data, missing some underlying patterns. |
The model is fairly robust but may benefit from further tuning or more training to improve its learning. |
High Test, Low Train |
An unusual scenario, potentially indicating data leakage or overestimation of test performance. |
The model’s performance is likely unreliable; investigate potential data issues or random noise. |
Moderate Test, High Train |
The model fits the training data well but doesn’t generalize as effectively, capturing only some test patterns. |
The model is slightly overfitting; adjustments may be needed to improve generalization on unseen data. |
Moderate Test, Moderate Train |
The model shows balanced but moderate performance on both datasets, capturing some patterns but missing others. |
The model is moderately fitting; further improvements could be made in both training and generalization. |
Moderate Test, Low Train |
The model underperforms on training data and doesn’t generalize well, leading to moderate test performance. |
The model may need more complexity, additional features, or better training to improve on both datasets. |
Low Test, High Train |
The model overfits the training data, performing poorly on the test set. |
The model doesn’t generalize well; simplifying the model or using regularization may help reduce overfitting. |
Low Test, Low Train |
The model performs poorly on both training and test datasets, failing to learn the data patterns effectively. |
The model is underfitting; it may need more complexity, additional features, or more data to improve performance. |
Recommendations for Publishing Results#
When publishing results, scenarios with moderate train and moderate test performance can be used for complex phenotypes or diseases. However, results showing high train and moderate test, high train and high test, and moderate train and high test are recommended.
For most continuous phenotypes, results typically fall in the moderate train and moderate test performance category.
2. Reporting Generalized Performance:#
One can also report the generalized performance by calculating the difference between the training and test performance, and the sum of the test and training performance. Report the result or hyperparameter combination for which the sum is high and the difference is minimal.
Example code:
df = divided_result.copy() df['Difference'] = abs(df['Train_best_model'] - df['Test_best_model']) df['Sum'] = df['Train_best_model'] + df['Test_best_model'] sorted_df = df.sort_values(by=['Sum', 'Difference'], ascending=[False, True]) print(sorted_df.iloc[0].to_markdown())
3. Reporting Hyperparameters Affecting Test and Train Performance:#
Find the hyperparameters that have more than one unique value and calculate their correlation with the following columns to understand how they are affecting the performance of train and test sets:
Train_null_modelTrain_pure_prsTrain_best_modelTest_pure_prsTest_null_modelTest_best_model
4. Other Analysis#
Once you have the results, you can find how hyperparameters affect the model performance.
Analysis, like overfitting and underfitting, can be performed as well.
The way you are going to report the results can vary.
Results can be visualized, and other patterns in the data can be explored.
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib notebook
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
divided_result['pvalue'] = 1
df = divided_result.sort_values(by='Train_best_model', ascending=False)
print("1. Reporting Based on Best Training Performance:\n")
print(df.iloc[0].to_markdown())
df = divided_result.copy()
# Plot Train and Test best models against p-values
plt.figure(figsize=(10, 6))
plt.plot(df['pvalue'], df['Train_best_model'], label='Train_best_model', marker='o', color='royalblue')
plt.plot(df['pvalue'], df['Test_best_model'], label='Test_best_model', marker='o', color='darkorange')
# Highlight the p-value where both train and test are high
best_index = df[['Train_best_model']].sum(axis=1).idxmax()
best_pvalue = df.loc[best_index, 'pvalue']
best_train = df.loc[best_index, 'Train_best_model']
best_test = df.loc[best_index, 'Test_best_model']
# Use dark colors for the circles
plt.scatter(best_pvalue, best_train, color='darkred', s=100, label=f'Best Performance (Train)', edgecolor='black', zorder=5)
plt.scatter(best_pvalue, best_test, color='darkblue', s=100, label=f'Best Performance (Test)', edgecolor='black', zorder=5)
# Annotate the best performance with p-value, train, and test values
plt.text(best_pvalue, best_train, f'p={best_pvalue:.4g}\nTrain={best_train:.4g}', ha='right', va='bottom', fontsize=9, color='darkred')
plt.text(best_pvalue, best_test, f'p={best_pvalue:.4g}\nTest={best_test:.4g}', ha='right', va='top', fontsize=9, color='darkblue')
# Calculate Difference and Sum
df['Difference'] = abs(df['Train_best_model'] - df['Test_best_model'])
df['Sum'] = df['Train_best_model'] + df['Test_best_model']
# Sort the DataFrame
sorted_df = df.sort_values(by=['Sum', 'Difference'], ascending=[False, True])
#sorted_df = df.sort_values(by=[ 'Difference','Sum'], ascending=[ True,False])
# Highlight the general performance
general_index = sorted_df.index[0]
general_pvalue = sorted_df.loc[general_index, 'pvalue']
general_train = sorted_df.loc[general_index, 'Train_best_model']
general_test = sorted_df.loc[general_index, 'Test_best_model']
plt.scatter(general_pvalue, general_train, color='darkgreen', s=150, label='General Performance (Train)', edgecolor='black', zorder=6)
plt.scatter(general_pvalue, general_test, color='darkorange', s=150, label='General Performance (Test)', edgecolor='black', zorder=6)
# Annotate the general performance with p-value, train, and test values
plt.text(general_pvalue, general_train, f'p={general_pvalue:.4g}\nTrain={general_train:.4g}', ha='left', va='bottom', fontsize=9, color='darkgreen')
plt.text(general_pvalue, general_test, f'p={general_pvalue:.4g}\nTest={general_test:.4g}', ha='left', va='top', fontsize=9, color='darkorange')
# Add labels and legend
plt.xlabel('p-value')
plt.ylabel('Model Performance')
plt.title('Train vs Test Best Models')
plt.legend()
plt.show()
print("2. Reporting Generalized Performance:\n")
df = divided_result.copy()
df['Difference'] = abs(df['Train_best_model'] - df['Test_best_model'])
df['Sum'] = df['Train_best_model'] + df['Test_best_model']
sorted_df = df.sort_values(by=['Sum', 'Difference'], ascending=[False, True])
print(sorted_df.iloc[0].to_markdown())
print("3. Reporting the correlation of hyperparameters and the performance of 'Train_null_model', 'Train_pure_prs', 'Train_best_model', 'Test_pure_prs', 'Test_null_model', and 'Test_best_model':\n")
print("3. For string hyperparameters, we used one-hot encoding to find the correlation between string hyperparameters and 'Train_null_model', 'Train_pure_prs', 'Train_best_model', 'Test_pure_prs', 'Test_null_model', and 'Test_best_model'.")
print("3. We performed this analysis for those hyperparameters that have more than one unique value.")
correlation_columns = [
'Train_null_model', 'Train_pure_prs', 'Train_best_model',
'Test_pure_prs', 'Test_null_model', 'Test_best_model'
]
hyperparams = [col for col in divided_result.columns if len(divided_result[col].unique()) > 1]
hyperparams = list(set(hyperparams+correlation_columns))
# Separate numeric and string columns
numeric_hyperparams = [col for col in hyperparams if pd.api.types.is_numeric_dtype(divided_result[col])]
string_hyperparams = [col for col in hyperparams if pd.api.types.is_string_dtype(divided_result[col])]
# Encode string columns using one-hot encoding
divided_result_encoded = pd.get_dummies(divided_result, columns=string_hyperparams)
# Combine numeric hyperparams with the new one-hot encoded columns
encoded_columns = [col for col in divided_result_encoded.columns if col.startswith(tuple(string_hyperparams))]
hyperparams = numeric_hyperparams + encoded_columns
# Calculate correlations
correlations = divided_result_encoded[hyperparams].corr()
# Display correlation of hyperparameters with train/test performance columns
hyperparam_correlations = correlations.loc[hyperparams, correlation_columns]
hyperparam_correlations = hyperparam_correlations.fillna(0)
# Plotting the correlation heatmap
plt.figure(figsize=(12, 8))
ax = sns.heatmap(hyperparam_correlations, annot=True, cmap='viridis', fmt='.2f', cbar=True)
ax.set_xticklabels(ax.get_xticklabels(), rotation=90, ha='right')
# Rotate y-axis labels to horizontal
#ax.set_yticklabels(ax.get_yticklabels(), rotation=0, va='center')
plt.title('Correlation of Hyperparameters with Train/Test Performance')
plt.show()
sns.set_theme(style="whitegrid") # Choose your preferred style
pairplot = sns.pairplot(divided_result_encoded[hyperparams],hue = 'Test_best_model', palette='viridis')
# Adjust the figure size
pairplot.fig.set_size_inches(15, 15) # You can adjust the size as needed
for ax in pairplot.axes.flatten():
ax.set_xlabel(ax.get_xlabel(), rotation=90, ha='right') # X-axis labels vertical
#ax.set_ylabel(ax.get_ylabel(), rotation=0, va='bottom') # Y-axis labels horizontal
# Show the plot
plt.show()
1. Reporting Based on Best Training Performance:
| | 0 |
|:-----------------|------------:|
| clump_p1 | 1 |
| clump_r2 | 0.1 |
| clump_kb | 200 |
| p_window_size | 200 |
| p_slide_size | 50 |
| p_LD_threshold | 0.25 |
| numberofpca | 6 |
| tempalpha | 0.1 |
| l1weight | 0.1 |
| Train_pure_prs | -408.191 |
| Train_null_model | 0.233856 |
| Train_best_model | 0.252479 |
| Test_pure_prs | -425.092 |
| Test_null_model | 0.137656 |
| Test_best_model | 0.156992 |
| pvalue | 1 |
2. Reporting Generalized Performance:
| | 0 |
|:-----------------|-------------:|
| clump_p1 | 1 |
| clump_r2 | 0.1 |
| clump_kb | 200 |
| p_window_size | 200 |
| p_slide_size | 50 |
| p_LD_threshold | 0.25 |
| numberofpca | 6 |
| tempalpha | 0.1 |
| l1weight | 0.1 |
| Train_pure_prs | -408.191 |
| Train_null_model | 0.233856 |
| Train_best_model | 0.252479 |
| Test_pure_prs | -425.092 |
| Test_null_model | 0.137656 |
| Test_best_model | 0.156992 |
| pvalue | 1 |
| Difference | 0.0954873 |
| Sum | 0.409471 |
3. Reporting the correlation of hyperparameters and the performance of 'Train_null_model', 'Train_pure_prs', 'Train_best_model', 'Test_pure_prs', 'Test_null_model', and 'Test_best_model':
3. For string hyperparameters, we used one-hot encoding to find the correlation between string hyperparameters and 'Train_null_model', 'Train_pure_prs', 'Train_best_model', 'Test_pure_prs', 'Test_null_model', and 'Test_best_model'.
3. We performed this analysis for those hyperparameters that have more than one unique value.
