Friday, July 31, 2015

BSDCan 2015 Trip Report: Mark Linimon


I arrived two days early so that I could recover from the first stage of my vacation preceding the conference, and then socialize.  (The latter can be an advantage because trying to carry on a conversation in the larger groups later in the week can sometimes be daunting.)  To some extent this was just a continuation of my vacation (i.e. funded out of my own pocket.)

Glen and Deb and I started off with an evening of conversation over dinner.  This was mostly a "well, where are we now?" discussion.  Glen and I adjourned later for a discussion of ports build infrastructure that eventually wound up with us toasting the memory of some departed four-legged friends.

As the various members of the Ports Management Team arrived over the next few days, I began spending time with them in various informal sessions.  Ports remains my main area of interest.

Wednesday was the first day of the devsummit.  I attended the nested kernels session which I will happily admit was over my head.  I was also interested to hear how Isilon manages their use of FreeBSD.

Thursday morning brought the documentation session by Warren Block. Warren has many new ideas. While translations are not my interest, Warren is thinking about ways to bring people into the translation project that involve finding ways to lower barrier of entry.  In particular, he is experimenting with technology that would allow non-technical users to contribute to a translation dictionary without having to have commit rights.  (Later, actual commits would be vetted.) I found persuasive the idea that we are excluding "people with an interest in documentation but no grasp of our documentation build processes" from participating.  This is an object lesson that my own viewpoint has been too narrow and based on my own past experience with various markup languages.

Thursday evening I was asked to sit in on the portmgr meeting.  We spent several hours going over "where are we now and where do we want to go".  Within the last 12 months we have finally acquired enough hardware to be able not just to build the bare-minimum packages and experimental (exp-) runs, but also to think about what our ideal build procedures should look like, and what kind of analysis tools that we need.  Some of those ideas are still under active discussion, but I think I can comment about the following:


  •  Establish some "publish criteria" before declaring a quarterly branch as the "new" official branch.  Right now there are no criteria other than trying to read through lists of error summaries. We need to be able to create some kind of automated figure-of-merit for each particular build.  My view is that portsmon could be augmented to help with this, but in any case, we need this functionality.

  • It is difficult to evaluate regressions on individual runs.  A "test instance" of portsmon should probably be used to do this.  A method already exists to display these data but even the portmgrs do not understand it well.  (FWIW, portsmon does not "know" what builds are -exp builds; it treats all build inputs the same.  It would be far easier to instantiate a separate "test" portsmon that takes input from those builds to display to those specifically interested, than to rearchitect the UI.  This would continue to keep those data out of the current instance, which is intended for the general public as well as ports infrastructure developers.)  (In fact, the development instance in Austin does exactly this, as well as gather occasional test results from my tier-2 machines here.  I am used to it, but others would most likely be confused.)

Friday brought the first day of BSDCan proper.  The key talk that I wanted to see was about QEMU package builds.  I retain an interest in the tier-2 architectures.  I believe that supporting multiple architectures helps keep FreeBSD more robust.  Sean Bruno and Stacey Son have made a great deal of progress on the cross-build poudriere environment.  Of particular interest was Stacey's list of what work remains to be done to complete the emulation.  There is, of course, a tradeoff curve between how complete the emulation of (e.g.) syscalls is, and how many real-world use cases are affected.  No emulation can ever be pefect, of course.

I also attended the afternoon network performance presentation.  I do not have enough in-depth knowledge of the network system to understand all the implications, but was interested to pursue whether some of the framework could help us to do more general performance testing such as Kris Kennaway used to do.  This is an area that I think we should spend effort on.

The most important session on Saturday was on packaging the base system. I expected this talk to be more controversial than it was.  (Perhaps everyone was beginning to get tired ...)

There is a great deal of work that has been done so far, but it seems there is still some distance yet to go.  One of the next problems to face will be to define "what do we consider a FreeBSD base system to be". However, there are already some variations on the theme, among the more notable being nanobsd, crochet, and freebsd-wifi-build.  IMHO each ofthese is an incompatible attempt to solve the same underlying problem. If it is possible to create one solution that will encompass all these attempts, it will be a big achievement.

Summary: lessons learned:

It's hard to overstate the importance of the "hallway track".  Among the people who I was able to reconnect with (including Glen as mentioned above) were Sean Bruno, Gavin Atkinson, Stacey Son, Justin Hibbits, and Marcel Moolenaar.  A lot of good ideas get kicked around there as well.

Another lesson that I learned the hard way last year is that it's physically impossible for me to attend every single session and then every hacking lounge and every nightly social activity.  I was much more judicious in pacing myself and this helped me not wear out as early on as I did last year.

All in all I got a much better idea of what areas we need to work on for the rest of the year by attending the conference.

mcl



Friday, July 24, 2015

FreeBSD 10.2-RC1 Now Available

The first RC build of the 10.2-RELEASE cycle is now available.

Installation images are available for the amd64, i386, ia64, powerpc, powerpc64, and sparc64 architectures.

FreeBSD/arm SD card images are available for the BEAGLEBONE, CUBOX-HUMMINGBOARD, GUMSTIX, RPI-B, PANDABOARD, and WANDBOARD kernels.

FreeBSD 10.2-RC1 is also available on several third-party hosting providers.

See the PGP-signed announcement email for installation image checksums and more information.

Friday, July 17, 2015

FreeBSD 10.2-BETA2 Now Available

The second BETA build of the 10.2-RELEASE cycle is now available.

Installation images are available for the amd64, i386, ia64, powerpc, powerpc64, and sparc64 architectures.

FreeBSD/arm SD card images are available for the BEAGLEBONE, CUBOX-HUMMINGBOARD, GUMSTIX, RPI-B, PANDABOARD, and WANDBOARD kernels.

FreeBSD 10.2-BETA2 is also available on several third-party hosting providers.

See the PGP-signed announcement email for installation image checksums and more information.

Saturday, July 11, 2015

FreeBSD 10.2-BETA1 Now Available

The first BETA build of the 10.2-RELEASE cycle is now available.

Installation images are available for the amd64, armv6, i386, ia64, powerpc, powerpc64, and sparc64 architectures.

Additionally, FreeBSD 10.2-BETA1 is available on several third-party hosting providers.

See the PGP-signed announcement email for installation image checksums and more information.

Wednesday, July 8, 2015

BSDCan 2015 Trip Report: Shonali Balakrishna

One of the slides at the opening session of the BSDCan conference read, “So many smart people in one room. This is why I attend BSD conferences.” I smiled to myself, and began to soak up the excitement of my first BSD conference. After nine talks and multiple conversations with the aforementioned smart people, I am thoroughly convinced! Not only does a BSD conference have way too many very smart people in one room, but also some of the nicest. You hear about the great work done by the FreeBSD community all the time, but not enough is said of the incredible FreeBSD community and the inspiration it provides.

I came in on 12th June, the first day of the scheduled talks, after a grueling finals week at grad school, and was very sleep deprived, but the excitement and the energy at the conference kept me wide awake. I found each of the nine talks I attended very interesting. It was incredible listening to THE Steven Bourne talk about the creation of the Bourne shell at the plenary, and his many insights and inside stories to do with it. I especially loved the part where he explained how he convinced Dennis Ritchie to inculcate the ‘void’ type into C and his candor about the debugging of shell scripts.

I especially enjoyed the following talks, among the ones I attended:

FreeBSD Operations at LimeLight Networks ­ Content Delivery Networks (CDN) are something that interests me, having had a research component on it at graduate school this year, and to see it in the operations perspective, with their usage of FreeBSD was very interesting. They deploy their own backbone and use FreeBSD on their backbone. For a big CDN, with many data centers, large scale installations of FreeBSD on the edge is an important factor. Such an operations workload requires fluidity in software and configuration changes and what is interesting is that only a handful of people are involved in the design and operations at Limelight Networks, as opposed to the usual model. The usage of FreeBSD allows both of these as you are pulled into the source tree and can get involved in the operations of the system. While deploying FreeBSD, their strategies involve upstreaming everything, using ports, and creating a src team. Tools used by Limelight were discussed, which I found informative ­ Zabbix for monitoring to ensure API driven configuration management, monitoring in test, dev and QA to form an efficient feedback loop, OpenTSDB for storing time series metrics, SaltStack for configuration management using a declarative style where the system takes action based on policy using an orchestration bus, Vagrant for pushing FreeBSD on the edge servers.

CloudABI by Ed Schouten ­ which is a Unix application binary interface which provides capability based security. It extends the concept of Capsicum, by providing a more compact representation with only about 60 system calls. This can be applied to cloud computing environments, instead of using full system virtualization or virtualized namespaces. CloudABI can be used in a far simpler way, with better performance, without complicated configurations, while providing security by controlling access to sockets, files and other resources. This was interesting to me, because it provides a whole new way of creating environments by building on capability based security to provide isolation and resource access control at the process level rather than complete isolation of environments to provide the level of security required.

An Introduction to the Implementation of ZFS by Kirk McKusick ­ This was an interesting talk for more reasons than the fact that it was Kirk McKusick himself talking about ZFS ­ there was also a fire alarm going off in the middle of the talk, followed by the arrival of fire trucks! I loved this talk for the clarity with which Dr. McKusick explained the ZFS block structure and checkpointing and made simple the rather complex concepts like freeing of filesystem and snapshot blocks in ZFS. The closing session hosted by Dan Langille was a lot of fun, and again the awesomeness of the BSD community was on shining display. I also attended the closing party at the nearby Lowertown Brewery, which was fun and a good opportunity for some networking as well.

Through this conference, it was great to finally meet Gavin Atkinson, who was the administrator for all of us GSOC students at FreeBSD last summer. I also got to converse with Diane Bruce, Ed Schouten, Deb Goodkin, Justin Gibbs, David Chisnall among others and every conversation was refreshingly unique in the perspectives it provided me with about the work done at FreeBSD, specific projects in FreeBSD, technology in general, the codebase, ways to contribute to FreeBSD, history, music and other myriad subjects. Ottawa was an incredible city to visit, and is so steeped in history and culture, and I soaked in the museums, art galleries and beautiful hiking trails it provided.

In terms of the work I’d like to do for FreeBSD in the future, besides wrapping up the testing of the BSNMP IPv6 project, I also learnt more about other projects I could potentially contribute to in the future, like further IPv6 support (userland cleanup, unification of ping and ping6, unification of traceroute and traceroute6), improvements to the Bluetooth module, 802.11 improvements and Space Communication Protocol, after exploring these with the people I spoke to. Even though I landed at Ottawa mentally exhausted, I returned completely excited and inspired, with many plans for future contributions to FreeBSD, and so very grateful to the FreeBSD Foundation for having provided me with this opportunity to attend my first (of many more, hopefully) BSDCan.

Shonali Balakrishna

BSDCan 2015 Trip Report: Kamil Czekirda

My trip to Ottawa, Canada was sponsored by The FreeBSD Foundation and took place in June 2015, where I attended FreeBSD devsummit and BSDCan 2015. I would like to thank FreeBSD Foundation for sponsoring my trip, Gavin Atkinson for an invitation to devsummit, and the mentor of my project, Warner Losh, for the meeting.

I arrived to Ottawa on Tuesday evening, I found the residence, checked in and dropped off my luggage. I had to look for a shop and do some shopping. It was too late for lounging about so I stayed for the rest of the day at the residence.

The first day of the Developer Summit / Tutorials for me started with “FreeBSD Storage for Sysadmin” tutorial by Michael W. Lucas. I experienced the jet lag, but it was stronger than I expected... During the brake I met Maciej Pasternacki from my country. Interestingly, Maciej was my neighbor on the plane from Montreal to Ottawa;) After the tutorial I joined the devsummit. My working group was “Designing Universal Configuration Files for FreeBSD” chaired by Allan Jude. There was a very interesting discussion about dividing system utilities for targets to USL­ify and not to ucl. I like the idea “show running config” for single compiled config for the entire system. In the end of the first devsummit / tutorial day we moved to the hacker lounge at the University residence, then during dinner I met Zbigniew Bodek, the third person from Poland at BSDCan.

On the second day of the Developer Summit / Tutorials for me started from the next tutorial about DNSSEC, unfortunately I missed Documentation and ARMv8 working groups, but I couldn’t be everywhere. After the Group Photo I joined the OpenZFS working group.

The main conference started on Friday. It was officially opened by Dan Langille, where Deb Goodkin told about the FreeBSD Foundation. The next step was an amazing keynote by Stephen Bourne about the history of Unix and /bin/sh. After the keynote we finished the devsummit track. Then I attended the Public track, where Kylie Lang explained why Microsoft loves FreeBSD and then Warren Block criticized the installers. The last subject was especially interesting for me, because I worked on installers last summer. During the break I was talking with Warren about profiles for installers. The next talk I've seen, was a legendary “A reimplementation of NetBSD using a MicroKernel” by Andy Tanenbaum, I heard very positive opinions about this talk on Sofia last year.

The lunch break was very productive, except the lunch itself, of course. I found Kris Moore and told him about my patch for freebsd­update. He directed me to Colin Percival and finally with Allan Jude and Nathan Whitehorn supported Xin Li who closed bug 194746 in FreeBSD Bugzilla.

Then there was George Neville-­Neil's talk about measuring network performance. After this talk people tried a cake brought by The FreeBSD Foundation, they were celebrating 15th birthday. Other talks I've seen were from the System Administration track, where Joseph Mingrone explained how scientist use FreeBSD in computation.

And the last one of that day ­ the Olivier Cochard­-LabbĂ©’s talk entitled “Large­ Scale Plug & Play x86 Network Appliance Deployment Over Internet” ­ was one of the most interesting ones for me. There was another event of the day ­ after dinner in the hacker lounge I had a meeting with the mentor of my GSoC project – Warner Losh.

The last day of the conference for me started with the second George Neville-­Neil's presentation about teaching advanced operating systems. I’m a student and it was very interesting how they do it in Cambridge. The next talk was given by Randi Harper about fighting harassment with open source tools. We really need more non­-tech talks on conferences.

After lunch I attended the Daichi Goto’s talk: “FreeBSD for High Density Servers”. Daichi told how people from NEC install thousands of FreeBSD by PXE. The last two talks were about virtualization, the first given by Michael Dexter briefly described virtualization possibilities in FreeBSD, and the other presented by Maciej Pasternacki showed a lightweight virtualization technique based on FreeBSD jails and ZFS. During his talk we had a building evacuation, after which Maciej finished his talk about Jetpack.

I was really surprised by the charity auction that took place in the time of closing session, where Dan gave a great show. I had a great time during the closing party in a brewery in the Market, I’ve tried bacon in maple sauce for the first time.

The last point of the schedule was the Tourist Day, which we started at 10.00 AM, I’ve never seen Ottawa before, so I really enjoyed this short, but very interesting trip. We finished at about 2.00 PM and after dinner I came back and rented a bike for one more trip.

I think that attending the conferences like those is a huge motivation for work for new people. It was a great opportunity to meet people whom I had known only from the Internet. I hope I will be able to participate in devsummits and BSD conferences again in the future.

Kamil Czekirda

Tuesday, July 7, 2015

BSDCan 2015 Trip Report: Christian Brueffer

For several years now, BSDCan has been known in the FreeBSD universe as the place to be. It features great talks, is attended by many people FreeBSD core developers, some who usually don't make it over to one of the European conferences, as well as the largest FreeBSD developer summit.  In 2010 I was lucky to have the FreeBSD Foundation sponsor my trip to BSDCan.  This year I was fortunate to have the Foundation sponsor my trip again.

I arrived in Canada some time before the conference to see a bit of the country, and to shed any sign of jetlag before the start of the developer summit (a lesson I learned from my previous BSDCan trip).  At the "Goat BoF" the day before the start of the developer summit I met many familiar faces, and was able to meet some people I knew by name but had never met in person before.

The next day was marked the start of the devsummit.  It's hard to find a better place to get things done than in this inspiring atmosphere, single-mindedly focused on FreeBSD. Uninterrupted FreeBSD time is something I usually lack, so I managed to make a progress on a few of my long-standing TODO items, most notably the export of the OpenBSM repository from the FreeBSD Perforce server to git.  The issues this presented on previous attempts were still there, but this time I was able to get it done and, after some more work after the conference, the repository has finally moved to GitHub.  Another thing I explored was using BHyVe as a provider for Vagrant, a popular tool for creating reproducible development environments.

On the second day of the summit, I participated in the documentation working group.  It was great to finally meet some of the most active people in the last few years.  We had a productive day with Warren Block as ring leader, discussing topics ranging from translation, over the doc toolchain, to manpages.  A special treat was the presence of Ingo Schwarze from the OpenBSD project and maintainer of the mandoc package, as well as Ryan Lortie from GNOME. Having outside experiences definitely benefited the discussion.  One of my goals was discussing moving the release documentation from the source repository to the doc repository, to allow us to make corrections after a release.  It was a good discussion, and Hiroki Sato explained some issues I had not considered beforehand.

The conference itself was a fantastic.  It does not happen every day that you have people like Stephen Bourne, Andrew Tanenbaum and Kirk McKusick under the same roof.  There were many great talks, however the most interesting one for me was "Molecular Evolution, Genomic Analysis and FreeBSD", given by Joseph Mingrone.  I'm a graduate student in cancer genomics (the analysis of genetic data from cancer patients), which is a closely related field with many of the same issues and tools.  Biological and medical research is dominated by Linux, with the problem that scientific researchers usually don't write the best and most portable code.  This makes it more difficult to use FreeBSD, as oftentimes one wants to try out software from the latest research paper. Combined with the fact that biological/medical datasets are oftentimes huge, performance really matters, ruling out virtualization.  As it turns out, Joseph had the same experience but found that bhyve appears to have low enough overhead to make this feasible.  My hope is that we can work together to make FreeBSD a better platform for this kind of research.

In the evenings, I spent time in the hacker and doc lounges where the discussions and the work continued.

All in all, the conference and the devsummit were a great success for me.  I learned a lot, and it's always astonishing how seeing the cool stuff other people have been working on motivates me to work on FreeBSD better myself.  Thank you to the FreeBSD Foundation for making this trip possible!

Christian Brueffer