Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WARNING Cannot find normalisation 'XXX' #169

Open
moxqiang opened this issue Sep 27, 2023 · 3 comments
Open

WARNING Cannot find normalisation 'XXX' #169

moxqiang opened this issue Sep 27, 2023 · 3 comments

Comments

@moxqiang
Copy link

Dear FAN-C providers:
I wanted to use fancplot to plot a Hic map, but I encountered an error.

2023-09-27 17:33:32,993 INFO Using non-interactive backend /public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py:353: UserWarning: Support for Juicer .hic v9 is still in beta. Please report any issues to https://github.com/vaquerizaslab/fanc/issues/92 warnings.warn(f"Support for Juicer .hic v{self.version} is still in beta. " /public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py:353: UserWarning: Support for Juicer .hic v9 is still in beta. Please report any issues to https://github.com/vaquerizaslab/fanc/issues/92 warnings.warn(f"Support for Juicer .hic v{self.version} is still in beta. " 2023-09-27 17:33:33,732 INFO Found 1 regions /public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py:797: UserWarning: Cannot find normalisation vector for chromosome: HPV16, normalisation: SCALE, resolution: 500000, unit: BP. This could indicate that SCALE normalisation did not work for this chromosome. Will return NaN instead. res=resolution, unit=unit /public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py:797: UserWarning: Cannot find normalisation vector for chromosome: HPV18, normalisation: SCALE, resolution: 500000, unit: BP. This could indicate that SCALE normalisation did not work for this chromosome. Will return NaN instead. res=resolution, unit=unit /public/home/qqfu/miniconda3/envs/fan-c/bin/fancplot:189: UserWarning: There was an error with plot 0, region chr2:0-240000000 (Traceback (most recent call last): File "/public/home/qqfu/miniconda3/envs/fan-c/bin/fancplot", line 169, in __init__ fig, axes = gf.plot(plot_region) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/plotter.py", line 211, in plot p.plot(r) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/base_plotter.py", line 785, in plot plot_output = self._plot((x_region, y_region)) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 299, in _plot self._refresh(region) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 342, in _refresh self.current_matrix = self.hic_buffer.get_matrix(*region) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 143, in get_matrix self._BUFFERING_STRATEGIES[self.buffering_strategy](self, *regions) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 186, in _buffer_relative norm=self.norm, oe=self.oe, log=self.log) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/architecture/comparisons.py", line 587, in matrix m_bottom = self.matrix_bottom.matrix(*args, **kwargs) * self.scaling_factor File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/matrix.py", line 1025, in matrix **kwargs) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/matrix.py", line 963, in regions_and_matrix_entries row_regions, col_regions, edges_iter = self.regions_and_edges(key, *args, **kwargs) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/matrix.py", line 869, in regions_and_edges row_regions = list(row_regions) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py", line 900, in _region_subset norm = self.normalisation_vector(region.chromosome) File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py", line 789, in normalisation_vector raise ValueError("Cannot find normalisation '{}'".format(normalisation)) ValueError: Cannot find normalisation 'SCALE' ) tb)) 2023-09-27 17:33:34,785 WARNING Cannot find normalisation 'SCALE'

Here is my code:
fancplot chr2:0-240mb -o ./SiHa.500kb.ic.png -p split -l /public/home/qqfu/HPV-PoreC/HiPore-C/Use.hg38_hpv16_18.Res-02/Result/normal.PoreC.hic/SiHa.norm.hic@500kb /public/home/qqfu/HPV-PoreC/HiPore-C/Use.hg38_hpv16_18.Res-02/Result/hic.hicpro/SiHa_HiC.allValidPairs.hic@500kb

My Hi-C data was processed using the "addNorm" method in Juicer.
code:
java -Xmx20g -jar /public/home/qqfu/software/juicer/juicer_tools.2.20.00.jar addNorm $HIC_FILE -j 20 -k IC -w 1000

Many thanks for your help!

@moxqiang
Copy link
Author

Sorry for my format:

2023-09-27 18:30:25,870 INFO Using non-interactive backend
/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py:353: UserWarning: Support for Juicer .hic v9 is still in beta. Please report any issues to #92
warnings.warn(f"Support for Juicer .hic v{self.version} is still in beta. "
/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py:353: UserWarning: Support for Juicer .hic v9 is still in beta. Please report any issues to #92
warnings.warn(f"Support for Juicer .hic v{self.version} is still in beta. "
2023-09-27 18:30:26,631 INFO Found 1 regions
/public/home/qqfu/miniconda3/envs/fan-c/bin/fancplot:189: UserWarning: There was an error with plot 0, region chr2:0-240000000 (Traceback (most recent call last):
File "/public/home/qqfu/miniconda3/envs/fan-c/bin/fancplot", line 169, in init
fig, axes = gf.plot(plot_region)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/plotter.py", line 211, in plot
p.plot(r)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/base_plotter.py", line 785, in plot
plot_output = self._plot((x_region, y_region))
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 299, in _plot
self._refresh(region)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 342, in _refresh
self.current_matrix = self.hic_buffer.get_matrix(*region)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 143, in get_matrix
self._BUFFERING_STRATEGIES[self.buffering_strategy](self, *regions)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/plotting/hic_plotter.py", line 186, in _buffer_relative
norm=self.norm, oe=self.oe, log=self.log)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/architecture/comparisons.py", line 568, in matrix
m_top = self.matrix_top.matrix(*args, **kwargs)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/matrix.py", line 1025, in matrix
**kwargs)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/matrix.py", line 963, in regions_and_matrix_entries
row_regions, col_regions, edges_iter = self.regions_and_edges(key, *args, **kwargs)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/matrix.py", line 869, in regions_and_edges
row_regions = list(row_regions)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py", line 900, in _region_subset
norm = self.normalisation_vector(region.chromosome)
File "/public/home/qqfu/miniconda3/envs/fan-c/lib/python3.7/site-packages/fanc/compatibility/juicer.py", line 789, in normalisation_vector
raise ValueError("Cannot find normalisation '{}'".format(normalisation))
ValueError: Cannot find normalisation 'SCALE'
)
tb))
2023-09-27 18:30:26,681 WARNING Cannot find normalisation 'SCALE'

@kaukrise
Copy link
Collaborator

For v0 Juicer files, FAN-C tries to use the SCALE norm by default. You can specify your normalisation manually:

https://fan-c.readthedocs.io/en/latest/fanc-executable/compatibility.html#juicer-hic-files

In your post you mention using addNorm with -k IC, so I would assume that this should work:

fancplot chr2:0-240mb -o ./SiHa.500kb.ic.png -p split -l /public/home/qqfu/HPV-PoreC/HiPore-C/Use.hg38_hpv16_18.Res-02/Result/normal.PoreC.hic/SiHa.norm.hic@500kb@IC /public/home/qqfu/HPV-PoreC/HiPore-C/Use.hg38_hpv16_18.Res-02/Result/hic.hicpro/SiHa_HiC.allValidPairs.hic@500kb@IC

But the normalisation might be called differently internally. If it does not work, see this post for a way to get the available normalisations in your object.

@moxqiang
Copy link
Author

Thank you for your help.

I tested the initial Hi-C files generated using different workflows, and found that the options "none", "VC", "VC_SQRT", and "NONE" worked fine, but I encountered the same problem with "KR" and "ICE".

Additionally, when using data processed by Juicer, only the "NONE" option worked for visualization, while other options did not.

I'm wondering if this is because my workflow is not compatible with FAN-C (where one dataset is generated using the complete HiCPro workflow and the other dataset is generated using Juicer's pair conversion).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants