Grzegorz Kossakowski gave a tech talk on his Scala+GWT project that is now online.
Slides are available here.
The talk was around September, and there has been a lot of progress since then. For example, the system can now compile and run a much larger subset of Scala.
Wednesday, February 2, 2011
Does IPv6 have hope?
That probably sounds like a strange question. 6 > 4, and IPv6 is simply the newer version of IPv4. Everyone will eventually be upgraded, right?
That's not really accurate. IPv6 is not backwards compatible with IPv4. Despite the name, it's really a different protocol. From a technical perspective, asking if we will all upgrade to IPv6 is similar to asking if we will all sidegrade to IPX. The only difference is in the branding. IPv6 sounds like an upgrade.
Dan Bernstein has a great article up on the technical issues with migrating to IPv6. In general, the plans that IPv6 advocates are discussing involve every node on the Internet upgrading to support IPv6 and IPv4 simultaneously, and then we can make a big switchover. At first blush, this sounds like a good plan. It worked for HTTP, and as Bernstein points out, it worked for MX records for SMTP servers. However, so far at least, IPv6 isn't designed to work that way.
The problem is that, unlike with these other examples, it's not a simple software upgrade to simultaneously support IPv6 and IPv4. Most distressingly, every node on the Internet needs to additionally have an IPv6 address. That alone is a fatal flaw. It's simply not going to happen. With the MX record transition, nodes without an MX record simply fell back on their A record, which they already had. With the HTTP transition, HTTP/1.1 was and remains an optional extension. Every node can always fall back to HTTP/1.0 or even HTTP/0.9. With IPv6, however, any node that doesn't have an IPv6 address simply doesn't get to play.
The same thing is happening with DNS. In addition to allocating all those IPv6 addresses, you need to add them to DNS. Until every DNS record is updated to have both IPv6 and IPv4 addresses in it, it's not possible to flip the switch.
As a separate issue, does anyone really care about the other features of IPv6 other than the extended address space? IPv6 comes with a host of features, and some of the more complicated and computationally expensive ones like IPsec are mandatory. These strike me as things that would better as optional extensions, and indeed, most of these features except the larger address space are already being explored as such.
Overall, the main thing that IPv6 brings us over IPv4 is the larger address space. Why not make that an optional extension, too? The IP packet format already allows for extensions. Routers trying to forward an extended-address packet could simply ping each other before doing it, and if the next router in the chain doesn't support them, bounce back a "host not reachable" packet. Lots of software would need updating, e.g. the socket APIs used on end point software would need to support optional, longer addresses. However, the changes would be much smaller than are required to support IPv6.
So if the best transition to big address spaces is to extend IPv4 rather than stage a simultaneous leap to IPv6, what is going to happen? One possibility is that such an IPv4++ will be designed, it will be rebranded as IPv6, and everyone will simply ignore the failed experiments with a more radically different IPv6. Another possibility is that such an IPv4++ will be rebranded as IPv7. This is all branding and politics, though I must say that the most honest thing would be to simply call it IPv4.
Another possibility is that everyone interested in big addresses will get suckered into the IPv6 quagmire, and it just won't happen. This isn't clearly a bad thing. The "small" address space of IPv4 is plenty large if we continue to have an Internet that is a patchwork of interconnected networks rather than a true globally controlled network. With the smaller address space, my coffee maker can still send a packet to your coffee maker, but from each coffee maker's perspective it will be sending packets out into the cloud. That seems healthy, to me. Perhaps you want to support virtual coffee makers, or to have transparent coffee maker failover. It's not really my business exactly how you route my packet in your network. Why would you want to export a node address to me that tries to pinpoint a specific machine?
A more exotic possibility is that some other network gains market share. Some network that offers real, pressing advantages, unlike IPv6. A packet-switching network protocol is like a social network or an instant messaging system. New networks take over old ones by initially offering something attractive enough that people will operate on both networks simultaneously. Once enough people are on the new network, they can start taking it for granted, and the old network can deflate in usage very rapidly. Really, though, what possible improvement would the packet switching layer have that would encourage that initial batch of people to use the new one in parallel to the old one? Clearly it's nothing in the laundry list of features in IPv6, because adoption has been really tepid. Further, all the really good network improvements have been possible to retrograde onto IPv4.
My best guess is that we continue on with IPv4 plus extensions. More tentatively, I would guess that we never get around to extending it for larger address spaces. If larger address spaces do become a pressing concern, however, I'd expect IPv4 to be extended rather than for the whole world to waste time on switching to completely new protocol. It's just good engineering.
That's not really accurate. IPv6 is not backwards compatible with IPv4. Despite the name, it's really a different protocol. From a technical perspective, asking if we will all upgrade to IPv6 is similar to asking if we will all sidegrade to IPX. The only difference is in the branding. IPv6 sounds like an upgrade.
Dan Bernstein has a great article up on the technical issues with migrating to IPv6. In general, the plans that IPv6 advocates are discussing involve every node on the Internet upgrading to support IPv6 and IPv4 simultaneously, and then we can make a big switchover. At first blush, this sounds like a good plan. It worked for HTTP, and as Bernstein points out, it worked for MX records for SMTP servers. However, so far at least, IPv6 isn't designed to work that way.
The problem is that, unlike with these other examples, it's not a simple software upgrade to simultaneously support IPv6 and IPv4. Most distressingly, every node on the Internet needs to additionally have an IPv6 address. That alone is a fatal flaw. It's simply not going to happen. With the MX record transition, nodes without an MX record simply fell back on their A record, which they already had. With the HTTP transition, HTTP/1.1 was and remains an optional extension. Every node can always fall back to HTTP/1.0 or even HTTP/0.9. With IPv6, however, any node that doesn't have an IPv6 address simply doesn't get to play.
The same thing is happening with DNS. In addition to allocating all those IPv6 addresses, you need to add them to DNS. Until every DNS record is updated to have both IPv6 and IPv4 addresses in it, it's not possible to flip the switch.
As a separate issue, does anyone really care about the other features of IPv6 other than the extended address space? IPv6 comes with a host of features, and some of the more complicated and computationally expensive ones like IPsec are mandatory. These strike me as things that would better as optional extensions, and indeed, most of these features except the larger address space are already being explored as such.
Overall, the main thing that IPv6 brings us over IPv4 is the larger address space. Why not make that an optional extension, too? The IP packet format already allows for extensions. Routers trying to forward an extended-address packet could simply ping each other before doing it, and if the next router in the chain doesn't support them, bounce back a "host not reachable" packet. Lots of software would need updating, e.g. the socket APIs used on end point software would need to support optional, longer addresses. However, the changes would be much smaller than are required to support IPv6.
So if the best transition to big address spaces is to extend IPv4 rather than stage a simultaneous leap to IPv6, what is going to happen? One possibility is that such an IPv4++ will be designed, it will be rebranded as IPv6, and everyone will simply ignore the failed experiments with a more radically different IPv6. Another possibility is that such an IPv4++ will be rebranded as IPv7. This is all branding and politics, though I must say that the most honest thing would be to simply call it IPv4.
Another possibility is that everyone interested in big addresses will get suckered into the IPv6 quagmire, and it just won't happen. This isn't clearly a bad thing. The "small" address space of IPv4 is plenty large if we continue to have an Internet that is a patchwork of interconnected networks rather than a true globally controlled network. With the smaller address space, my coffee maker can still send a packet to your coffee maker, but from each coffee maker's perspective it will be sending packets out into the cloud. That seems healthy, to me. Perhaps you want to support virtual coffee makers, or to have transparent coffee maker failover. It's not really my business exactly how you route my packet in your network. Why would you want to export a node address to me that tries to pinpoint a specific machine?
A more exotic possibility is that some other network gains market share. Some network that offers real, pressing advantages, unlike IPv6. A packet-switching network protocol is like a social network or an instant messaging system. New networks take over old ones by initially offering something attractive enough that people will operate on both networks simultaneously. Once enough people are on the new network, they can start taking it for granted, and the old network can deflate in usage very rapidly. Really, though, what possible improvement would the packet switching layer have that would encourage that initial batch of people to use the new one in parallel to the old one? Clearly it's nothing in the laundry list of features in IPv6, because adoption has been really tepid. Further, all the really good network improvements have been possible to retrograde onto IPv4.
My best guess is that we continue on with IPv4 plus extensions. More tentatively, I would guess that we never get around to extending it for larger address spaces. If larger address spaces do become a pressing concern, however, I'd expect IPv4 to be extended rather than for the whole world to waste time on switching to completely new protocol. It's just good engineering.
Monday, January 31, 2011
That scary Internet
National governments are coming to fear the Internet as a potentially disruptive mechanism for their publics. China and Australia have installed national firewalls to attempt to filter information crossing their borders to and from the greater Internet. Most recently, Egypt has recently shut down portions of its Internet infrastructure.
Many reports speak of the Egyptian shut down as a done deal. However, this is a misleading viewpoint. In point of fact, many Egyptians are still connected to the Internet through various means. The Internet is architected so that packets can take any route available from their source IP to their destination IP. As the old saying goes, "The Net interprets censorship as damage and routes around it". Like with so many other things, an official shut down just shuts down official business. Criminals don't care, nor do most of the general public.
Regarding the American kill switch, I must wonder how the discussion has gotten as far as it did given American politics. Aside from being technically hopeless, and for making times of peace more dangerous, it just doesn't seem American to let the president shut down a major category of speech. Has there ever been a U.S. president that tried to get a media kill switch, i.e. the ability to shut down every newspaper, pamphlet, printer, and copying machine at the press of a button?
Overall, I expect this gradual creeping oversight to know no bounds. The U.S. government is ham-handed, its members would universally prefer not to be discussed, and units such as the FCC are seeking a new reason to exist. Instead of gradually fighting each individual effort as they attempt to chip away at the open Internet, I would prefer a categorical principle that the U.S. government just does not have authority over the Internet. There's no reason they should, and they're not even competent.
Many reports speak of the Egyptian shut down as a done deal. However, this is a misleading viewpoint. In point of fact, many Egyptians are still connected to the Internet through various means. The Internet is architected so that packets can take any route available from their source IP to their destination IP. As the old saying goes, "The Net interprets censorship as damage and routes around it". Like with so many other things, an official shut down just shuts down official business. Criminals don't care, nor do most of the general public.
Regarding the American kill switch, I must wonder how the discussion has gotten as far as it did given American politics. Aside from being technically hopeless, and for making times of peace more dangerous, it just doesn't seem American to let the president shut down a major category of speech. Has there ever been a U.S. president that tried to get a media kill switch, i.e. the ability to shut down every newspaper, pamphlet, printer, and copying machine at the press of a button?
Overall, I expect this gradual creeping oversight to know no bounds. The U.S. government is ham-handed, its members would universally prefer not to be discussed, and units such as the FCC are seeking a new reason to exist. Instead of gradually fighting each individual effort as they attempt to chip away at the open Internet, I would prefer a categorical principle that the U.S. government just does not have authority over the Internet. There's no reason they should, and they're not even competent.
Thursday, January 6, 2011
Software patents help what, again?
Via James Robertson, I read that Interval is suing about a dozen major software companies over patent infringement. I am having trouble finding an original link to the case information, but here's a link to one copy of Interval's opening volley.
Here's the IP Interval is suing over:
The theory behind patents is that, without patent protection, nobody would have invented the idea in question. By offering patent protection, companies will devote resources to research that they otherwise would not have. Can anyone seriously believe, however, that we would have more innovation if all of AOL, Apple, eBay, Facebook, Google, Netflix, Office Depot, OfficeMax, Staples, Yahoo, or YouTube had honored this patent and not shown similar items on their web sites? Does anyone believe that if Interval hadn't "invented" this idea, that nobody else would have?
An additional part of the rationale for patents is that the idea are difficult to develop, that they would only emerge if significant private resources were dedicated to its research. That, too, is hard to believe for this idea. How long did it take the guys at Interval to come up with this idea? Five minutes, maybe?
I have an idea how to stimulate the software industry. Stop issuing software patents.
Here's the IP Interval is suing over:
The ’507 patent describes an invention that enables a user to efficiently review a large body of information by categorizing and correlating segments of information within the body of information and generating displays of segments that are related to the primary information being viewed by the user.From this alone, you might thing they have some advanced technique for categorizing and showing related information. No, they really are claiming that the whole idea of showing users a list of items related to the one they are looking at is an Interval invention. For example, here is their complaint about eBay:
Defendant eBay has infringed and continues to infringe one or more claims of the ’507 patent under 35 U.S.C. § 271. eBay operates the eBay.com and Half.com websites, which provide content such as product listings and advertisements to users. In order to help users find additional content that may be of interest, the software and hardware that operate these websites compare the available content items to determine whether they are related. When a user views a particular content item, the eBay.com and Half.com websites generate displays of related content items so as to inform the user that the related items may be of interest. For example, as demonstrated by Exhibit 8, when a user views a particular product listing on eBay.com, the eBay.com website displays both the selected product information (identified by the orange box) and links to other related products (identified by the green boxes). The hardware and software associated with the eBay websites identified above and any other eBay websites that perform this function infringe at least claims 20, 21, 22, 23, 24, 27, 28, 31, 34, 37, 63, 64, 65, 66, 67, 70, 71, 74, 77, and 80 of the ’507 patent under 35 U.S.C. § 271.
The theory behind patents is that, without patent protection, nobody would have invented the idea in question. By offering patent protection, companies will devote resources to research that they otherwise would not have. Can anyone seriously believe, however, that we would have more innovation if all of AOL, Apple, eBay, Facebook, Google, Netflix, Office Depot, OfficeMax, Staples, Yahoo, or YouTube had honored this patent and not shown similar items on their web sites? Does anyone believe that if Interval hadn't "invented" this idea, that nobody else would have?
An additional part of the rationale for patents is that the idea are difficult to develop, that they would only emerge if significant private resources were dedicated to its research. That, too, is hard to believe for this idea. How long did it take the guys at Interval to come up with this idea? Five minutes, maybe?
I have an idea how to stimulate the software industry. Stop issuing software patents.
Monday, January 3, 2011
References without page numbers
James Robertson asks how we can reference a part of a book, if we read the book on a Kindle or other electronic medium:
It's an old problem, though, and it has a lot of old solutions. It comes up any time the same text is printed multiple times with different page numbers. Two examples would be codes of law and the Christian Bible. If you want to cite a part of one of these, it's poor form to use a page number, because that page number is only valid for a specific printing. You instead make reference to the detailed numbers that have been applied to the sub-entities of the text.
Going forward, it would help if books started containing more fine-grained numberings as a matter of course. In theory we could instead use character count or word counts, but that has two problems. It is prone to differences in convention, e.g. how many characters is a paragraph indent, and how many words are in counter-revolutionary. Worse, it doesn't work well for people using the print version, who would need a specially printed version with the position counts on the bottom of each page or in the margins.
Bill Venners foresaw this problem for Programming in Scala, and he was careful to publish the ebook version such that it has the exact same page numbers as the printed book. This is possible because the ebook is a PDF file, and PDF files have the same pagination on every device. In addition to the consistent page numbering, the book includes fine-grained number of all the sections, figures, tables, and larger programming listings, so you can also cite things that way. In short, feel free to copiously cite parts of Programming in Scala. Don't worry about the ebook readers--they'll be able to look up your references just fine.
...what does a page number even mean? It should be simple to graft the physical form page number into the metadata, but as we go forward, there may well be books for which no physical form exists. What then?This isn't a new problem, but it's exacerbated by current norms of book publishing. Printed books often don't number their sub-entities at a finer grained level than chapters, so if you don't have the physical version in front of you, all you can cite is the chapter. Worse, if someone else has a physical version, and you're reading the electronic version, it's problematic if they give you a cite for a page number.
It's an old problem, though, and it has a lot of old solutions. It comes up any time the same text is printed multiple times with different page numbers. Two examples would be codes of law and the Christian Bible. If you want to cite a part of one of these, it's poor form to use a page number, because that page number is only valid for a specific printing. You instead make reference to the detailed numbers that have been applied to the sub-entities of the text.
Going forward, it would help if books started containing more fine-grained numberings as a matter of course. In theory we could instead use character count or word counts, but that has two problems. It is prone to differences in convention, e.g. how many characters is a paragraph indent, and how many words are in counter-revolutionary. Worse, it doesn't work well for people using the print version, who would need a specially printed version with the position counts on the bottom of each page or in the margins.
Bill Venners foresaw this problem for Programming in Scala, and he was careful to publish the ebook version such that it has the exact same page numbers as the printed book. This is possible because the ebook is a PDF file, and PDF files have the same pagination on every device. In addition to the consistent page numbering, the book includes fine-grained number of all the sections, figures, tables, and larger programming listings, so you can also cite things that way. In short, feel free to copiously cite parts of Programming in Scala. Don't worry about the ebook readers--they'll be able to look up your references just fine.
Wednesday, December 22, 2010
That's not how it works
James Robertson shares this depressing quote from the FCC:
Follow the link for analysis.
Let me focus just on this part. The FCC, here, joins the ranks of those who think the Internet is a star topology. The apparent model is that there's an Internet, and then everyone plugs their computer into the Internet. When one user routes a packet to another user, it takes two hops: one to the center node, and one to the other user. Everything that happens within this mythical center node is abstracted away.
As an aside, the FCC also presents a view of the Internet where a handful of providers are sending broadcasts to the masses. Individuals don't contract for Internet services. They are "consumers", and they "subscribe" to the feeds. Leave that aside for now.
The Internet is not a star topology, but a general network. When you send a packet to someone else, it usually takes a dozen or two hops to get to them. How fast it gets to them depends enormously on the intermediate nodes that are taken along the way. I used to play around with traceroute and watch just what routes the packets take under various circumstances. I saw some particularly striking examples when I worked on a Department of Defense bulletin board and watched how packets route between a university network and a DoD machine. Let's just say the routes favored security over latency. They'd go a LONG way in order to go through carefully controlled choke points.
Because the Internet works this way, people who provide Internet services work hard to make sure their servers are well connected with respect to their users. For example, if you want to provide service to British folks, then you really want to get a server up on the island. It wasn't so long ago that all major ftp sites had clones in the UK. Sending data across the English Channel, much less the Atlantic Ocean, was just horrendously slow. When you install an extra server in the UK, you must pay for it.
Relocating a server is just one option. It's also possible to lease network connections between where your server is and where you want the IP traffic to route to. When you do that, you will have to pay whomever you are leasing the bandwidth from.
In short, if you want better connectivity, you have to pay for it. The more you pay, the better the connectivity you get. What the FCC calls a disturbing development is a hair split away from how things already work. They seem to be riding on the notion of whether you pay a broadband provider or some other entity. I fail to see what a big difference it makes.
Let's try a few thought experiments and compare them to the star-topology model. Suppose Netflix pays Comcast to let them install some servers in the same building as a major Comcast hub. Is anything wrong with that? I don't see why. They'll get better bandwidth, but they're paying for all the expenses. Similarly, suppose Netflix, on their own dime, installs new network fiber from their data center to a major Comcast hub. Is there anything wrong with that? Again, I don't see it. After Netflix lays that network, would there be anything wrong with Comcast plugging into it and routing traffic to and from it? Again, I can't see how it would help users for them to decline.
Where the FCC seems to draw the line is when you go past barter and use more fungible resources. What if, instead of Netflix installing new network fiber itself, it pays Comcast to do it. And what if, instead of Comcast laying new fiber for each customer, they split the cost over different customers, giving more access to those who pay more. From the FCC's view, this goes from totally normal to something they've never seen in the past. From my view, this is how things work already. You pay more to get more bandwidth.
I wish the FCC would just abandon trying to regulate Internet service. I want a neutral network, but I don't see how the FCC is going to anything but hurt. I want the Internet we have, not something like broadcast TV, cable, wired telephony, or cellular telephony. I don't think it is a coincidence that the Internet is both less regulated and far more neutral than these other networks.
"A commercial arrangement between a broadband provider and a third party to directly or indirectly favor some traffic over other traffic in the connection to a subscriber of the broadband provider (i.e., 'pay for priority') would raise significant cause for concern," the Commission then elaborates. This is because "pay for priority would represent a significant departure from historical and current practice."
Follow the link for analysis.
Let me focus just on this part. The FCC, here, joins the ranks of those who think the Internet is a star topology. The apparent model is that there's an Internet, and then everyone plugs their computer into the Internet. When one user routes a packet to another user, it takes two hops: one to the center node, and one to the other user. Everything that happens within this mythical center node is abstracted away.
As an aside, the FCC also presents a view of the Internet where a handful of providers are sending broadcasts to the masses. Individuals don't contract for Internet services. They are "consumers", and they "subscribe" to the feeds. Leave that aside for now.
The Internet is not a star topology, but a general network. When you send a packet to someone else, it usually takes a dozen or two hops to get to them. How fast it gets to them depends enormously on the intermediate nodes that are taken along the way. I used to play around with traceroute and watch just what routes the packets take under various circumstances. I saw some particularly striking examples when I worked on a Department of Defense bulletin board and watched how packets route between a university network and a DoD machine. Let's just say the routes favored security over latency. They'd go a LONG way in order to go through carefully controlled choke points.
Because the Internet works this way, people who provide Internet services work hard to make sure their servers are well connected with respect to their users. For example, if you want to provide service to British folks, then you really want to get a server up on the island. It wasn't so long ago that all major ftp sites had clones in the UK. Sending data across the English Channel, much less the Atlantic Ocean, was just horrendously slow. When you install an extra server in the UK, you must pay for it.
Relocating a server is just one option. It's also possible to lease network connections between where your server is and where you want the IP traffic to route to. When you do that, you will have to pay whomever you are leasing the bandwidth from.
In short, if you want better connectivity, you have to pay for it. The more you pay, the better the connectivity you get. What the FCC calls a disturbing development is a hair split away from how things already work. They seem to be riding on the notion of whether you pay a broadband provider or some other entity. I fail to see what a big difference it makes.
Let's try a few thought experiments and compare them to the star-topology model. Suppose Netflix pays Comcast to let them install some servers in the same building as a major Comcast hub. Is anything wrong with that? I don't see why. They'll get better bandwidth, but they're paying for all the expenses. Similarly, suppose Netflix, on their own dime, installs new network fiber from their data center to a major Comcast hub. Is there anything wrong with that? Again, I don't see it. After Netflix lays that network, would there be anything wrong with Comcast plugging into it and routing traffic to and from it? Again, I can't see how it would help users for them to decline.
Where the FCC seems to draw the line is when you go past barter and use more fungible resources. What if, instead of Netflix installing new network fiber itself, it pays Comcast to do it. And what if, instead of Comcast laying new fiber for each customer, they split the cost over different customers, giving more access to those who pay more. From the FCC's view, this goes from totally normal to something they've never seen in the past. From my view, this is how things work already. You pay more to get more bandwidth.
I wish the FCC would just abandon trying to regulate Internet service. I want a neutral network, but I don't see how the FCC is going to anything but hurt. I want the Internet we have, not something like broadcast TV, cable, wired telephony, or cellular telephony. I don't think it is a coincidence that the Internet is both less regulated and far more neutral than these other networks.
Friday, December 17, 2010
Every paper and book on our laptops?
Dick Lipton speculates on that question:
I agree for Citeseer, and I agree for the local library. Very soon, if not already, we will have laptops that can hold the entirety of Citeseer and the entirety of the local library's collection of books. I was impressed when I looked at the file sizes for Project Gutenberg. Shakesperean plays take a few tens of kilobytes, and the largest archive they supply is a dual-sided DVD with over 29,000 books. I still remember the shock when I looked at a directory listing on their web site and the file sizes looked so small I thought the software must be broken.
As an aside, I wish I could say that the Association for Computing Machinery thought this way. Their current thinking that they'll have an online digital library that they take a toll on. If they really wanted to help science, they'd mail you a pre-indexed thumb drive you can load into your laptop and have all papers up until that date. I would bet that someone in physics works this out long before the ACM does. Who knows, though.
All this said, papers and books are backward looking. Nowadays, papers and books are developed as electronic content and then, only at delivery time, printed onto paper. An increasing amount of interesting material is simply never printed at all. Want a copy of the Scala Language Specification? It's essentially a book, but you won't find it at the local library. Over time, printed word is becoming a niche application. You only need it for reading something in depth, or if you want to physically hand it to someone. For the former, print on demand works more and more frequently, and for the latter, the number of times it happens is decreasing. As well, electronic ink just keeps getting better.
From the perspective of interesting words, as opposed to printed papers and books, it will take longer before personal computers can hold all the, ahem, material that is out there. It includes not just papers and books written by mathematicians, but also forum messages, blog posts, and even Facebook and Twitter messages written by all manner of people. Perhaps even then we are already at the point where our machines have enough storage, but it's certainly a lot more data than just for Citeseer and the library.
Of course, most people are only interested in a tiny fraction of all that information. Perhaps Dick Lipton really only cares about math papers from famous mathematicians. If the precise data interesting to someone can be identified, then the storage requirements for keeping a personal copy are much more reasonable, and in fact we probably are already there. However, identifying that subset of the data is, in general, entirely non-trivial.
Today there are applications like Citeseer that contain about one million papers. The total storage for this is beyond the ability of most of us to store on our laptops. But this should change in the near future. The issue is that the number of papers will continue to grow, but will unlikely grow as fast as memory increases. If this is the case then an implication is that in the future we could have all the technical papers from an area on our own devices. Just as realtime spelling is useful, realtime access to technical papers seems to be a potentially exciting development.[...]
Right now there are too many books, even restricted to a subfield like mathematics, to have all of them stored on a single cheap device. But this large—huge—amount of memory could easily become a small one in the future.
I agree for Citeseer, and I agree for the local library. Very soon, if not already, we will have laptops that can hold the entirety of Citeseer and the entirety of the local library's collection of books. I was impressed when I looked at the file sizes for Project Gutenberg. Shakesperean plays take a few tens of kilobytes, and the largest archive they supply is a dual-sided DVD with over 29,000 books. I still remember the shock when I looked at a directory listing on their web site and the file sizes looked so small I thought the software must be broken.
As an aside, I wish I could say that the Association for Computing Machinery thought this way. Their current thinking that they'll have an online digital library that they take a toll on. If they really wanted to help science, they'd mail you a pre-indexed thumb drive you can load into your laptop and have all papers up until that date. I would bet that someone in physics works this out long before the ACM does. Who knows, though.
All this said, papers and books are backward looking. Nowadays, papers and books are developed as electronic content and then, only at delivery time, printed onto paper. An increasing amount of interesting material is simply never printed at all. Want a copy of the Scala Language Specification? It's essentially a book, but you won't find it at the local library. Over time, printed word is becoming a niche application. You only need it for reading something in depth, or if you want to physically hand it to someone. For the former, print on demand works more and more frequently, and for the latter, the number of times it happens is decreasing. As well, electronic ink just keeps getting better.
From the perspective of interesting words, as opposed to printed papers and books, it will take longer before personal computers can hold all the, ahem, material that is out there. It includes not just papers and books written by mathematicians, but also forum messages, blog posts, and even Facebook and Twitter messages written by all manner of people. Perhaps even then we are already at the point where our machines have enough storage, but it's certainly a lot more data than just for Citeseer and the library.
Of course, most people are only interested in a tiny fraction of all that information. Perhaps Dick Lipton really only cares about math papers from famous mathematicians. If the precise data interesting to someone can be identified, then the storage requirements for keeping a personal copy are much more reasonable, and in fact we probably are already there. However, identifying that subset of the data is, in general, entirely non-trivial.
Subscribe to:
Posts (Atom)