Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Fix handling of timestamps in BigQuery insert data method#1250

Merged
dhermes merged 2 commits intogoogleapis:masterfrom
dhermes:fix-1247
Dec 1, 2015
Merged

Fix handling of timestamps in BigQuery insert data method#1250
dhermes merged 2 commits intogoogleapis:masterfrom
dhermes:fix-1247

Conversation

@dhermes
Copy link
Contributor

@dhermes dhermes commented Nov 30, 2015

Fixes #1247.

@dhermes dhermes added the api: bigquery Issues related to the BigQuery API. label Nov 30, 2015
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Nov 30, 2015
@dhermes dhermes changed the title Fix handling of timestamps in Bigquery insert data method Fix handling of timestamps in BigQuery insert data method Nov 30, 2015
@dhermes
Copy link
Contributor Author

dhermes commented Nov 30, 2015

I just realized Bigtable has me trained too much. Wrote Bigquery instead of BigQuery in the commit messages.

@dhermes
Copy link
Contributor Author

dhermes commented Nov 30, 2015

FWIW I cherry-picked the second commit onto head and ran the system test with the error reported by @kamatama41 and got the following:

======================================================================
ERROR: test_load_table_then_dump_table (bigquery.TestBigQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../gcloud-python/system_tests/bigquery.py", line 232, in test_load_table_then_dump_table
    rows, _, _ = table.fetch_data()
  File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/bigquery/table.py", line 611, in fetch_data
    rows_data = _rows_from_json(response.get('rows', ()), self._schema)
  File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/bigquery/_helpers.py", line 81, in _rows_from_json
    row_data.append(converter(cell['v'], field))
  File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/bigquery/_helpers.py", line 42, in _datetime_from_json
    return _datetime_from_microseconds(1e6 * float(value))
  File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/_helpers.py", line 235, in _datetime_from_microseconds
    return _EPOCH + datetime.timedelta(microseconds=value)
OverflowError: date value out of range

----------------------------------------------------------------------
Ran 11 tests in 69.796s

@tseaver
Copy link
Contributor

tseaver commented Nov 30, 2015

LGTM. OK to merge w/ pylint failing on Travis, pending #1248.

@dhermes
Copy link
Contributor Author

dhermes commented Nov 30, 2015

I'll just wait for #1248

As stated in the docs
> BigQuery stores TIMESTAMP data internally as a
> UNIX timestamp with microsecond precision
> ...Specifies the number of seconds since the epoch.
@dhermes
Copy link
Contributor Author

dhermes commented Dec 1, 2015

@tseaver Assuming your LGTM still holds after rebasing.

dhermes added a commit that referenced this pull request Dec 1, 2015
Fix handling of timestamps in BigQuery insert data method
@dhermes dhermes merged commit b4e7534 into googleapis:master Dec 1, 2015
@dhermes dhermes deleted the fix-1247 branch December 1, 2015 00:59
@kamatama41
Copy link

Great job! Thank you for fixing quickly 😄

@dhermes
Copy link
Contributor Author

dhermes commented Dec 1, 2015

Sure thing. Thanks for reporting and finding the bug.

parthea pushed a commit that referenced this pull request Mar 9, 2026
* chore(deps): update all dependencies

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants