Query example:

select 1 as TAG,
       null as PARENT,
       '0' as "key", 7.27 as "value"
for json explicit

Result:

[["0",7.26999999999999881]]

As a result, value should be as in the original 7.27, the format of the field is numeric (12, 5)

[["0",7.27]]

I'll be grateful, cast, round, truncnum doesn't help

asked 21 Nov, 15:46

Aleksandr%20Kushnirov's gravatar image

Aleksandr Ku...
264
accept rate: 0%

edited 21 Nov, 15:50


JSON numbers are either integer or double-precision floating point. Values like 7.27 cannot be expressed exactly as a floating point number. One option is to process these values as strings.

permanent link

answered 21 Nov, 23:07

Chris%20Keating's gravatar image

Chris Keating
6.2k38104
accept rate: 30%

I agree when the string value is - works correctly. But then you have to convert to floating point number when building in Google Charts. And this is extra time. Thank you for your answer, I will try

(22 Nov, 02:37) Aleksandr Ku...
Replies hidden
2

The problem is that at the end of the day, 2.7 cannot be accurately stored in base2 as a real number (just like 1/3 can't be exactly converted to base10) - you will always being dealing with some sort of approximation. Floating point numbers are a computer thing not a mathematical thing!

I think decimal datatypes (when available) are stored as integer values and the appropriate offset then applied.

(22 Nov, 05:57) Justin Willey
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×23
×5

question asked: 21 Nov, 15:46

question was seen: 75 times

last updated: 22 Nov, 05:57