Will Scott

Tag: tech

  • Public & Private

    I have ended up spending more time than I was expecting over that last few months struggling to figure out where I stand on how much of ones life should be public versus private. I’ve found myself being more private over the last year, somewhat to my dismay, and yet have felt increasingly uncomfortable being more open or publicizing fully what I’m working on. I think more than ever before I foresee my decisions forcing me down a more polarized path than I’ve previously realized.

    And yet, the topic that I’ve been privately working on is exactly this issue – building systems for public communication. The core of the issue boils down to the fear that public association with anonymity and privacy issues will lead to increased surveillance and travel restrictions. At the same time, a more fatalistic voice says that I am already easily linked with privacy issues through my digital footprint, and as such I am failing to promote my work without protecting myself from retribution. As such, I am a knowing participant in the “chilling effects” of surveillance, taking fears of travel restrictions and life disruptions as motivations against talking more about privacy and censorship.

    This unsettling association around the effects of working on privacy is seemingly pervasive. One indicator is how quickly we attempt to distance ourselves from the vocabulary. Research doesn’t attempt to circumvent censorship, but rather uses “adversary-resilient” protocols to handle “network interference.” I feel like I have unconsciously compensated by working on Code for Seattle, a great, uncontroversial, project supporting civic tech and my local community. Academic papers measuring the behavior of the Internet have been published anonymously presumably due to similar discomfort to what I feel.

    My path so far has led me to a ridge that is now falling off precipitously on both sides. To one side is public advocacy of circumvention systems. Down that side are the realizations of the fears above, difficulty traveling, difficulty presenting myself as unbiased, and general polarization. This seems hugely unfortunate: I bear no particular ill will towards the countries I’ve been to, and would love to continue having the ability to travel and define who I am. To the other side is the lure of anonymity, starting over and creating a second identity for sensitive work distanced from ‘me’. Unfortunately, while the lure seems appealing, I believe it is also unachievable. I have seen only a single instance of a ‘successful’ anonymous online persona: that of satoshi, the bitcoin creator.

    And so what is left is to continue balancing on that ridge while embracing a diversity of projects so that I can’t be easily labeled. I’m still heading the same direction I set in college and have used to navigate through graduate school: My goal is to make the Internet better.

  • Redstar Pictures

    redstar thumb

  • Continuing Security

    I took the next step on recommended security practices by upgrading to https-only, and enabling spdy on the server. Still don’t have optimal cipher strengths, which will require some additional algorithm preference tweaking.

  • RedStar

    redstar

    I got around to taking screenshots of RedStar 3, which hasn’t really made its way onto the internet yet. It really does look mac-like, and is using a skin that as far as I can tell is custom made rather than being modified from an existing Linux theme. Apps are packaged using a mac-like folder structure as well, with ‘contents’ and ‘Resources’ folders within the application folder.

  • Pretty excited about the data visualization course at UW this winter.

  • Code for Seattle meetup this evening. Big plans for the year.

  • Muxed Earth

  • Experimenting with 802.11s

    The purpose of this document is an explanation of what is defined in the IEEE 802.11s standard, and the states of current implementations based on this standard. 802.11s is the standardization of mesh networking. That is to say that it defines a protocol for mobile devices where they can communicate amongst themselves even when not directly connected, and can route data beyond their immediate environment.
    This work has been ongoing for close to a decade, and has been progressing as a single proposal for the last four. The standard is for a complete system, explaining the MAC level standard that data will be transferred. This includes how clients can securely create connections, and how and when they should resend packets they receive which are meant for other devices in the mesh. As a result, this standard has become effectively all encompassing, as it defines standards from the hardware signal levels all the way up to complex software routing and security.
    Even today, the standard is not formalized, meaning that nothing is focalized, and implementers are really very cautious to begin implementing mesh networking, because it may change and they would then have to re-write or design their implementation (it means specifically that wireless cards are not being built with mesh networking in mind, since chip design is more expensive and less changeable then software drivers.) The lowest level changes are made to the wireless frames in order to allow hops within the mesh, and would benefit from hardware changes to make those changes quickly implemented, however they will still work even when those changes are processed entirely in software, which is the current choice implementations are making until the standard is formalized.
    Routing can be specified either explicitly, as the tree that the source is aware of that will extend to the root (or final destination in the mesh) or ad-hoc, where intermediate nodes make their own decisions on how to route the packet. However, routing is done using a QOS-esque process, where the route is first established, and subsequent packets can use the same path until it needs to be recalculated due to topological changes. In addition there is a periodic discussion about which node in the mesh should act as a root, and the ordering of nodes, which is based upon which node has wired access to other networks, and tie-broken by the MAC addresses. Beyond this there are different levels of structure that these networks can take on, similar to the differences we see today in ad-hoc networks and networks with access-points. These are called meshes with and without registration. Without registration, any node can come and go, and the rest of the mesh is unaware of any topology beyond that directly adjacent to it. In registration mode, the root is aware of the entire tomography of the network, and can make informed choices about how routes should be created.
    While the major protocol specification seems to not be changing much at the present, there is still enough flux in details that implementations are not compatible with each other, and are still very rudimentary. I looked at three different implementations, none of which are inter-compatible, and all claim to be based off of the standard.
    The first is the implementation of mesh networks used in the OLPC. This was one of the first implementations of mesh networking, and is not compatible with any widely used implementation outside of those devices. It works in OLPC clusters, and is a technical feat, although since the central points that were intended to ground the mesh networks have not yet materialized, it is quite underutilized as I understand it. Instead, in the situations where OLPCs have been deployed, a standard 802.11b network typically works well since all of the machines are close enough to the central access point.
    The second is the open 802.11s project, which is designing a software driver for the Linux kernel. This version works currently for a limited set of wireless cards, whose Linux drivers are sufficiently open to allow the needed modifications. It is not included as part of the standard kernel distribution, which is a fairly powerful comment on the state of it’s development, since the kernel is typically thought of as including everything and the kitchen sink. It is simply the reference implementation being created as part of the development of the standard.
    The third implementation is available in the latest version of FreeBSD. This version is included in the distribution, and notes that it does not work with the Linux version due to how primitive they are. I was unable to test this implementation as I was hoping to, because the emulated lab I was using didn’t allow the permissions that the code wanted to make to the wireless drivers. However, this code promises to be one of the most practical current versions of mesh networking. Setup is performed through the standard ifconfig linux tool. Where one specifies the access point in other protocols, here they specify the mesh they wish to use, and the parameters associated with it. Once the mesh is specified, routes are automatically formed, and users can manually specify nodes they wish to block or favor using included implementations.
    I think the important takeaway from this exploration is into the state of mesh networking. I think the IEEE process here has been a hindrance to the development, and that it has trouble keeping up with the technological developments that are going on in this area. Enough new research keeps coming in that the standard needs to be constantly updated to keep from being outdated, and in so doing means that vendors are hesitant to implement inoperable devices, and keeps the whole field from the popularity it needs to really succeed. I think that we have had the opportunity to settle on an imperfect draft of this standard and actually have widespread mesh networking, but due to the fact that it can become useful only if everyone does the same thing, we have been very hesitant to do so.
    Hopefully as smart phones continue to evolve we will begin to see mesh networking standardized in those devices, as the potential it envelops becomes increasingly important to our communication. Given the issues we’ve seen in recent years with cell companies growing troubles keeping their access point bandwidth in line with the consumer demand feels unsustainable, and will hopefully result in an increasing desire for truly distributed communication for both better reliability and bandwidth.

  • on p2p in the browser

    The final project I want to accomplish this semester in my distributed computing independent study, is to come up with a way to access p2p capabilities from within the browser. I was originally hoping to do this with JavaScript API that relied behind the scenes on the new flash p2p protocol, rtmfp. The protocol has been around since last spring, but development for it is not exactly optimal. You need to be using flash to make the movie, and you need a proprietary and expensive piece of server software to coordinate data transfers. The protocol itself is still proprietary, meaning that Adobe won’t actually tell you how to make a server that can work with it, or how data is sent across the wire.

    They finally have started advertising it at least, and so it will gain some prominence, and eventually the details will come to light, but for now it is unrealistic to attempt to use flash for a general server agnostic service. (http://www.ietf.org/proceedings/10mar/slides/tsvarea-1.pdf)

    Instead, my plan is to modify the open source privoxy software and add p2p capabilities at that layer. My eventual goal is to make a communal grease-monkey system, where pages can be modified by the user through pieces of JavaScript, and those scripts can then be shared to friends. With the right level of abstraction, I think that this can produce a very powerful system. Starting next week, I’ll begin reading and hacking privoxy to find out how to integrate new code into that project.

  • Cornel Visit Days

    This is just a recounting of my visit days at cornell, so that I can keep it fresh in my head when I want to reflect on it in the future.

    Flight sucked – sat in pain with a migraine between Cleveland and Syracuse – it let up about when we landed, and I’m blaming it on mild food poisoning from the turkey sandwich I ate on the way to cleveland.

    The driver from the airport was a really nice guy, (me and another systems guy (wants to research planetary systems especially filesystems) got driven in at the same time) and pointed out all the interesting sites between Syracuse and Ithaca. He regularly drives for cornell events, so knew a lot about the place. Some random points of interest:
    * There is some skiing, but it’s not going to be comparable to stuff on the west coast
    * It’s a very definite bubble, which really doesn’t interact with the area around it
    * a fair amount of interaction with people from new york

    Hotel was nice, as much as you would want from a hotel. No problems getting checked in.
    Went down to the hotel bar where people were hanging out. talked to a couple interesting people:
    * A guy from princeton doing theory. didn’t have a ton of good things to say about princeton.
    * One of the sponsors (1st year) from istanbul who liked the small town and was used to it, but said she went in to new york a lot at the beginning to adjust. She is doing systems – works with weatherspoon and is doing a project trying to optimize the back end of archive.org
    * Another girl sponsor who is doing theory and seemed

    Breakfast wasn’t bad, they did it at their special hotel school. It was as good as you would need for breakfast.

    Campus tour was nice. Lots of fun stuff going on. really pretty place, better certainly than anywhere else I’ve visited. The gorges are pretty sweet, and the old buildings are super neat.

    Housing wasn’t that exciting – cheap, but unexciting. Especially the university stuff, but the guys house that we saw was pretty sparse, and not super nice either.

    Went back. the systems people that talked seemed quite well adjusted, and were all doing cool stuff. The other people interested in systems my year seemed a bit less so. The good news is that you get extra time to actually build systems, and it sounds like Gun is really into stuff and would be a good guy for me to get as an advisor in terms of productivity.

    the food wasn’t overpriced anywhere which is good.

    The party was pretty geeky, but at least people were drinking. Still, it devolved to rock band and taboo pretty quickly. I don’t know how you would really do that better.

    Overall is seems like a somewhat better mix of geeks and reasonable people than Mudd, but that isn’t saying that too much. I think I could do interesting research, and got really excited talking to van Renesse for dinner, but hakim and gun will need to be really cool and uw will need to be a bit disappointing for me to choose cornell. (it’s operating at a bit of a disadvantage)

    Sunday night at 11pm I realized that the schedule in my packet had my meeting with professors and started looking up their research areas

    Monday was really quite good as well.
    I got up at 7 something in order to take the bus over to the CS department lounge. There was fruit, coffee, bagels, and that sort of breakfast ready for us, which took about half an hour for people to get through, and then we got an introduction to the department in the main conference room by eva Tardos. (A cool lady) She works in theory, and just went over basic strengths of the department. It seems to be really theory heavy, but systems comes in second. Everyone TAs for 2 semesters, and then you’ll either keep doing it or move onto an RA ship once you find a project. Funding guaranteed, pretty high graduation rate. 5-6 years for systems folks, theory typically can get through in 5. You do quite a bit of course work in the first year or two and then move into research for the remainder. If you quit after 2-3 you leave with a masters. They were saying only 2-3 per class didn’t make it. 1-2 because they never found an advisor they actually connected with and 1-2 because they left for industry without ever writing up their thesis. (they don’t have much sympathy for the later type)

    The second talk was by the systems group. We went down to the systems lab (they’re going to replace it next year, but it doesn’t look that bad as it is. Certainly very spacious, and the cube setup is pretty productive – I like it from last summer) Mayers focuses on PLs a bit more than the rest of them, and he was talking for most of it. He has been writing java based compilers – extending the things in a java-like syntax – and actually has a full new language out of it. Called fabric. part of it’s claim to fame is that it has distributed objects that can be executed on remote hosts really easily. (he directs the DB toaster project I heard about sunday – which compiles down databases down to the needed queries.) He also does stuff with trust control.

    Then there were a set of three meetings.
    first was with Gun. He’s doing some really cool stuff. Mostly interested in flixQ which is a hybrid p2p content delivery service – you download movies like youtube, but the server coordinates to maximize global bandwidth. He’s also got his hands in a bunch of different places. One was a trusted operating system in order to make use of the trusted computing chips in modern computers. Another is looking at network measurements – i think? He has taught the advanced p2p class, so that’s his stuff as much as anything.

    Then robert van renesse. He talked about DHTs and other designs that sacrificed different things, in order to work better in different situations. The Kad system is good for fault tolerance, but you certainly have trouble always getting the most up-to-date copy, so it looses information potentially.
    Anyway, he had a system where you could have master-slave nodes for different parts of your ring and re-divide partitions on your ring as data or hosts increased. It’s mostly aimed at data center situations, where you still want all the distributed stuff kad gives you, but you need redundancy in a different way.

    Then was Hakim, the guy I’ve talked too before. We talked a bit about his work in filesystems, and the differences between UW and cornell.

    I talked to both Gun and Hakim about UW, since they both have done stuff there (hakim did undergrad, gun did his PHD). they both said that the two departments were the most comparable I would find. The real difference that I could get out of them was that they thought cornell went a bit further in formalizing their systems. That is – they said UW had a lot more industrial connections, and so could do lots of work improving and finding problems in existing systems. However, they cautioned that the problem is that just fixing the thing is problematic when you can’t theoretically back up your proof. (So, cornell they claim has more theoretical backing to their systems group, and does a better job proving optimality of their solutions) I’m waiting until I visit UW to trust them on that point.

    Lunch was at a chinese place halfway down the hill. oh yeah, cornell is at the top of a big hill, and we were staying in downtown at the bottom. “college Town” is where undergrads primarily hang out about halfway down the hill and really close to the undergrad dorms – that’s where this restaurant was. It wasn’t bad, and they seemed perhaps a bit more chinese than most americanized restaurants – probably good given that tons of grad students are going to be chinese.

    Talked more to one of the grad students about her work optimizing energy efficiency at Archive.org over lunch, and then talked with their physics post doc after lunch. This is project Ken, one of the older systems guys has been leading, where they’ve been working with cicso on core routing algorithms. This part right now is monitoring, where they’ve built a laser that can send valid packets through modulation at very precise timing. Then they can receive the packets and determine how the network is delaying and working with these packets. They found a lot more clumping than anyone was expecting, especially because most of that clumping is re-randomized by the end host in other simulations that have been done.

    Of interest here is that Dan is Mike’s younger brother – mike being the guy I was interested in working with a princeton.

    Then I sat through the theory presentation. One half of theory was people doing game-theory optimization stuff and solving big matrix problems (sci-comp). The other half was vision and graphics, which meant lots of pretty pictures. They’ve got lots of physics simulations to quickly and accurately render things like hair and yarn. THey also have a new vision guy who’s apparently super famous in the field named Noah. (at least all the vision graphics students were really excited about working with him) He’s the guy who’s been doing 3d reconstruction from photos on flickr and such. Microsoft has been working with him on photosynth (I think). He had some pretty visualizations of 3d reconstructed cities.

    After that there were three more meetings. i talked first with Fred first, he’s a security guy – and has advised and shaped government policy quite a bit. he’s also a very good figure skater. We talked about side-channel attacks, and how you could leak information from secure systems from things like timing and power monitoring. It was interested stuff, but he had a bit less to say about distributed systems than the other guys.

    Next was Mayers – who talked more about his java compiler and about systems to keep track of data flow through your programs so that you can figure out how many bits of information end up doing what you want (like optimizing randomness so you aren’t wasting entropy in converting distributions.)

    Finally was Ken Bergman? he talked more about the physics simulation and also talked about how cisco was planning to do more with them, and was willing to let them play with the next generation of internet routers at pretty much any level – he’s personally working on some fault tolerance applications. One of the things he also said though was that there was a huge potential here to build a new protocol to fix problems with IP so that a second set of global traffic could get sent around the country. His thoughts were to build a parallel system that would either have QOS guarantees or security guarantees. (e.g. only trusted parties are guaranteed to see/understand the data. The latter is a really cool problem, and is one where you really do need a new protocol at that low level, because if your messages are entirely encrypted while on the wire then you need to have the routers be able to somehow deal with that encryption.

    There was a poster session after that, where lots of grad students hung out and talked about what they were working on. i talked more about Gun’s work on hybrid p2p systems with the grad student who had built it with him, and talked with some of his other students.

    A group of us went out to dinner after that, I talked mostly with a guy from UT Austin who spent a gap year working at intel and is now wanting to do more theoretical stuff and stop doing the formal verification work that intel was having him do.

    In the evening we went out ice skating. I can skate well enough that I don’t fall down and don’t look like a complete klutz, but I’m certainly not great. The CS department has it’s own hockey team where grad students play the faculty. They were pushing that event as one of their big social activities.

    I got up this morning and caught a shuttle to Syracuse. My first leg to new york was with a guy from cal tech who’s interested in AI and graphics. He’s choosing between princeton and Cornell, and had a bunch of interesting things to say about cal tech.

    And now, I’m almost back to reality. It was a positive trip, and the ball is now in UW’s court to convince me that they’re better. There was a very laid back atmosphere in the department which I think would be fun to involve myself in, and the profs all were very approachable. All terms were on first-names, and I didn’t notice the east-coast formalism that I had been half-expecting.

    The two things that are the biggest cons are
    1. location – it’s hard to work with people outside of cornell, since that’s all that’s there.
    2. Housing – the housing looked pretty run-down – I think a city like seattle could get me a much newer / nicer apartment for not that much more money.