Author  Calculate calculator buffer size? 
Member
Posts: Location:
Joined: 01.01.70 Rank: Guest  
Today when I was in math class I came up with the thought "How many bytes can my calculator hold".
And after a while I came up with this: (2^8)^byte.
For example: (2^8)^2 = 65536, that's the biggest number you can put in 2 bytes.
My calculator gave me the overflow error on (2^8)^42.
Do you think this is right? Or is there any other way?


Author  RE: Calculate calculator buffer size? 
ynori7 Member
Posts: 1486 Location: #valhalla
Joined: 08.10.07 Rank: God  
How many digits can your calculator hold?
For example:
My calculator holds 10 decimal digits. That means the largest integer it can hold is 9999999999. Now, the rest depends on how your calculator handles decimal numbers. It may be that it just gives each digit a one byte, meaning it hold 10 bytes.
Or, if it needs to convert the decimal number to binary, 34 bits are necessary to represent the number 9999999999 in binary. However, 34 bits allow you to represent the number (2^34)1, which is more than 10 decimal digits. So if it's doing things that way, it cuts off precision based on the decimal number, not the binary.
EDIT:
(2^8)^2 = 65536, that's the biggest number you can put in 2 bytes.
The biggest number you can put in two bytes is 65535 (which is 2^16  1).
Edited by ynori7 on 220909 18:48 

Author  RE: Calculate calculator buffer size? 
Member
Posts: Location:
Joined: 01.01.70 Rank: Guest  
what level maths is this?


Author  RE: Calculate calculator buffer size? 
ynori7 Member
Posts: 1486 Location: #valhalla
Joined: 08.10.07 Rank: God  
Orillian wrote:
what level maths is this?
Binary? It's not generally taught in math classes. It's computer science.


Author  RE: Calculate calculator buffer size? 
Member
Posts: Location:
Joined: 01.01.70 Rank: Guest  
ynori7 wrote:
Orillian wrote:
what level maths is this?
Binary? It's not generally taught in math classes. It's computer science.
Just to be a dick...
Binary is not a level of math, just a number system. The same math from algebra to calculus can be applied to it. In this case its algebra so level 7th grade.


Author  RE: Calculate calculator buffer size? 
Member
Posts: Location:
Joined: 01.01.70 Rank: Guest  
not sure what that is in english, our year 7 is 12  13 i think

