Apache Arrows streams only ~5 million records out of ~7 million records in the table

[petr-ponomarenko]

When I use streaming of data from BQ table with 7 million records using Apache Arrows I get only about 5 million records. Other methods of getting data from that table into pandas work fine. I am following this example with one stream

python-bigquery-storage/samples/to_dataframe/main_test.py

Lines 130 to 139 in 6254bf2

	# This example reads from only a single stream. Read from multiple streams
	# to fetch data faster. Note that the session may not contain any streams
	# if there are no rows to read.
	stream = read_session.streams[0]
	reader = bqstorageclient.read_rows(stream.name)
	# Parse all Arrow blocks and create a dataframe. This call requires a
	# session, because the session contains the schema for the row blocks.
	dataframe = reader.to_dataframe(read_session)
	print(dataframe.head())

[tswast]

We probably need to update that code sample to explicitly request only one stream.

[tswast]

I've sent #114 which adds max_stream_count=1 to the create_read_session call. I believe the issue you're encountering is because additional streams are created, which are assigned the rows which you are missing.