Extraneous blank line printed to terminal on import.

[dwr-zroy]

• pyhecdss version: 1.1.6, Build py39_0
• Python version: Python 3.9.16
• Operating System: Windows 10, Build 19045.3324, 64 bit x86

Description

On import, the module prints a blank line to the terminal. Not typically an issue, but when module is imported in subprocesses, each import creates a new blank line on the terminal.

What I Did

Output

In a CMD window, I ran the following commands, and received the following outputs. Note the lack of a blank line between the other expressions.

Microsoft Windows [Version 10.0.19045.3324]
(c) Microsoft Corporation. All rights reserved.

C:\>mkdir test

C:\>cd test

C:\test>conda create -n pyhecdss_test -c cadwr-dms pyhecdss python=3.9.16 -q -y
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: C:\Users\zroy\AppData\Local\anaconda3\envs\pyhecdss_test

  added / updated specs:
    - pyhecdss
    - python=3.9.16


The following NEW packages will be INSTALLED:

  blas               pkgs/main/win-64::blas-1.0-mkl
  bottleneck         pkgs/main/win-64::bottleneck-1.3.5-py39h080aedc_0
  ca-certificates    pkgs/main/win-64::ca-certificates-2023.08.22-haa95532_0
  intel-openmp       pkgs/main/win-64::intel-openmp-2023.1.0-h59b6b97_46319
  mkl                pkgs/main/win-64::mkl-2023.1.0-h6b88ed4_46357
  mkl-service        pkgs/main/win-64::mkl-service-2.4.0-py39h2bbff1b_1
  mkl_fft            pkgs/main/win-64::mkl_fft-1.3.8-py39h2bbff1b_0
  mkl_random         pkgs/main/win-64::mkl_random-1.2.4-py39h59b6b97_0
  numexpr            pkgs/main/win-64::numexpr-2.8.4-py39h7b80656_1
  numpy              pkgs/main/win-64::numpy-1.25.2-py39h055cbcc_0
  numpy-base         pkgs/main/win-64::numpy-base-1.25.2-py39h65a83cf_0
  openssl            pkgs/main/win-64::openssl-3.0.10-h2bbff1b_2
  pandas             pkgs/main/win-64::pandas-2.0.3-py39h4ed8f06_0
  pip                pkgs/main/win-64::pip-23.2.1-py39haa95532_0
  pyhecdss           cadwr-dms/win-64::pyhecdss-1.3.0-py39_0
  python             pkgs/main/win-64::python-3.9.16-h1aa4202_3
  python-dateutil    pkgs/main/noarch::python-dateutil-2.8.2-pyhd3eb1b0_0
  python-tzdata      pkgs/main/noarch::python-tzdata-2023.3-pyhd3eb1b0_0
  pytz               pkgs/main/win-64::pytz-2022.7-py39haa95532_0
  setuptools         pkgs/main/win-64::setuptools-68.0.0-py39haa95532_0
  six                pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
  sqlite             pkgs/main/win-64::sqlite-3.41.2-h2bbff1b_0
  tbb                pkgs/main/win-64::tbb-2021.8.0-h59b6b97_0
  tzdata             pkgs/main/noarch::tzdata-2023c-h04d1e81_0
  vc                 pkgs/main/win-64::vc-14.2-h21ff451_1
  vs2015_runtime     pkgs/main/win-64::vs2015_runtime-14.27.29016-h5e58377_2
  wheel              pkgs/main/win-64::wheel-0.38.4-py39haa95532_0


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

C:\test>conda activate pyhecdss_test

(pyhecdss_test) C:\test>
(pyhecdss_test) C:\test>
(pyhecdss_test) C:\test>
(pyhecdss_test) C:\test>python
Python 3.9.16 (main, May 17 2023, 17:49:16) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyhecdss

>>>1+1
2
>>>None
>>>exit()

(pyhecdss_test) C:\test>
(pyhecdss_test) C:\test>
(pyhecdss_test) C:\test>
(pyhecdss_test) C:\test>conda deactivate
C:\test> conda remove -n pyhecdss_test --all -q -y

Remove all packages in environment C:\Users\zroy\AppData\Local\anaconda3\envs\pyhecdss_test:


## Package Plan ##

  environment location: C:\Users\zroy\AppData\Local\anaconda3\envs\pyhecdss_test


The following packages will be REMOVED:

  blas-1.0-mkl
  bottleneck-1.3.5-py39h080aedc_0
  ca-certificates-2023.08.22-haa95532_0
  intel-openmp-2023.1.0-h59b6b97_46319
  mkl-2023.1.0-h6b88ed4_46357
  mkl-service-2.4.0-py39h2bbff1b_1
  mkl_fft-1.3.8-py39h2bbff1b_0
  mkl_random-1.2.4-py39h59b6b97_0
  numexpr-2.8.4-py39h7b80656_1
  numpy-1.25.2-py39h055cbcc_0
  numpy-base-1.25.2-py39h65a83cf_0
  openssl-3.0.10-h2bbff1b_2
  pandas-2.0.3-py39h4ed8f06_0
  pip-23.2.1-py39haa95532_0
  pyhecdss-1.3.0-py39_0
  python-3.9.16-h1aa4202_3
  python-dateutil-2.8.2-pyhd3eb1b0_0
  python-tzdata-2023.3-pyhd3eb1b0_0
  pytz-2022.7-py39haa95532_0
  setuptools-68.0.0-py39haa95532_0
  six-1.16.0-pyhd3eb1b0_1
  sqlite-3.41.2-h2bbff1b_0
  tbb-2021.8.0-h59b6b97_0
  tzdata-2023c-h04d1e81_0
  vc-14.2-h21ff451_1
  vs2015_runtime-14.27.29016-h5e58377_2
  wheel-0.38.4-py39haa95532_0


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

C:\test>cd ..

C:\>del test /q

[dwr-zroy]

Using debug mode in VS Code Version 1.82.2, the extra line is being created at import by the call to: set_message_level(0) in __init__.py of pyhecdss.

This call to pyhecdss.set_message_level subsequently calls pyheclib.hec_zset.

Would adding a context manager (like contextlib.redirect_dtdout) within the set_message_level function be appropriate?

[dwr-psandhu]

@dwr-zroy Did you try the redirect stdout as you suggest above? If this is happening in the C/Fortran libraries then it is unlikely that python level redirection would work. Let me know if that works.

[dwr-zroy]

No I had not. I tested and that option didn't work as it was only redirecting the python level stdout as you pointed out. I found another solution that redirected the process level stdout by modifying the file descriptor at sys.__stdout__.

The solution below uses the smallest context managers I could drum up to accomplish the change with as little risk of modifying sys.__stdout__ outside of the context of the context manager. Two managers were needed to make sure devnull didn't stay open following an unexpected error.

I think a drawback to this solution is it's pretty verbose, and it is only used once. Additionally, I haven't explored whether or not errors raised in pyheclib still result in the correct errors in the calling python functions.

Branch on my fork for reference

pyhecdss.py with changes.

import collections
from . import pyheclib
import pandas as pd
import numpy as np
import os
import re
import time
import warnings
import logging
from datetime import datetime, timedelta
from calendar import monthrange
from dateutil.parser import parse
import sys
from contextlib import contextmanager
# some static functions

DATE_FMT_STR = '%d%b%Y'
_USE_CONDENSED = False


@contextmanager
def null_io():
    """Create a TextIOWrapper object pointing to os.devnull"""
    try: 
        with open(os.devnull, 'w') as null:
            yield null
    finally:
        pass


@contextmanager
def silent_std_out():
    """Small context manager to ignore stdout, even from FORTRAN subroutines. 
    
    Does not re-map stderr or stdin. Re-maps the system level stdout found 
    using `sys.__stdout__`. The following will produce no outputs to appear on
    the screen:

    >>> with silent_std_out():
    ...    print("Hello World")
    ...    call_noisy_subroutine()
    ...
    """
    # Clear pending, we want to see these
    sys.__stdout__.flush()
    with null_io() as null:
        try: 
            os.dup2(null.fileno(), sys.__stdout__.fileno())
            yield None
        finally:
            # Clear pending, we do not want to see these
            null.flush()
            os.dup2(sys.__stdout__t.fileno(), null.fileno())


def set_message_level(level):
    """
    set the verbosity level of the HEC-DSS library
    level ranges from "bort" only (level 0) to "internal" (level >10)
    """
    with silent_std_out():
        pyheclib.hec_zset('MLEVEL', '', level)


# The remainder of pyhecdss.py remains unchanged...