[GSoC] HAMMER2 block compression feature

[Justin Sherrill]

Daniel - One additional item that may be useful is learning something
about git, the revision control tool DragonFly (and many other
organizations) use.  You may already have encountered it or something
similar, but I don't see specific mention in your email.  It will be
the preferred way to store your code and to make it accessible for
others.

I can't tell you if there's a strong candidate for block compression -
the application period just started, so there's nothing yet.  Rather
than second-guess what's available, apply with a project proposal that
has what you most want to do.  That will do you the most good.

On Tue, Apr 23, 2013 at 3:14 PM, Daniel Flores <daniel5555 at gmail.com> wrote:
> Hello.
>
> My name is Daniel Flores and I’m a student from Spain. I study in Barcelona
> School of Informatics, which is a faculty of Polytechnic University of
> Catalonia, and it’s my last year before I graduate.
>
> I sent a mail with this proposal to Matthew 4 days ago, but I didn't get a
> reply yet, so I decided to send this to the mailing list, as it is suggested
> on Dragonfly's GSoC page. Before submitting any official application, I
> would like to ask for your opinion and criticism regarding my proposal and
> whether I have the required skill and adequate planning to successfully
> finish this project.
>
> I’m interested in implementing the block compression feature in HAMMER2,
> which is described on Dragonfly’s GSoC page [1]. The feature consists in
> that any written block should be compressed, if possible, to 50% of its size
> or less (in powers of 2 – 32 KB, 16 KB and so on, the initial size of block
> being 64 KB). It should be possible to set that compression mode on any
> directory, so that new contents written in it and its subdirectories would
> be compressed.
>
> I believe that I can succeed because I have programming experience in C/C++,
> since we use those languages for most practices in our university. I also
> completed courses centered on algorithms and their implementation in C or
> C++. I also have an interest in operating systems, their concepts and their
> development. I completed a course on Operating Systems development where I
> and a partner implemented a simple linux-based kernel called “ZeOS” (using C
> and, at times, x86 assembler). It’s a very simple kernel that has a task
> scheduler, system calls, exceptions, interruptions, memory management and a
> file system. We were able to successfully run it on bochs emulator and it
> passed all tests needed to successfully finish the course.
>
> If you wish, you may take a look at it [2].
>
> Sadly, it’s documented in Spanish, but I just want to provide it as an
> example of code that I can produce.
>
> In spite of it being a kernel implementation, I think that my skills and
> knowledge aren’t yet the best suitable for kernel development projects, so
> instead I prefer to do something moderately difficult, yet related to
> operating systems, like HAMMER2 block compression feature.
>
> My initial outline for project planning is the following:
>
> 1. (Starting May 27 or before – June 17) During community bonding period I
> would get myself familiarized with the community, Dragonfly BSD system/code
> and “hammer2” branch code. I would also refresh my knowledge on compression
> algorithms and would complete a design for the future implementation, like
> choosing the appropriate algorithm and having defined everything needed to
> be done. I have read Matthew's design document for HAMMER2 [3] and, as I
> understand, the algorithm #2 is not yet determined, so I believe this should
> be decided before the actual coding period will start. Also, more detailed
> planning for coding period would be done, with clear milestones and
> deliverables.
>
> The deliverables for this phase would be in the form of documents that would
> describe the design, the chosen algorithm(s) and the calendar with
> milestones.
>
> 2. (June 17 – July 29) I would start by coding a prototype application that
> would implement the defined design, consisting of algorithms #0, #1 and #2.
> I would need it to insure that the algorithm is implemented correctly and to
> do all necessary debugging and refining before integrating it in HAMMER2. I
> would create a series of tests to prove that the algorithm works well. If
> time allows, I would also try to study its behavior statistically and to
> improve it, if possible.
>
> The deliverables would be code of the prototype as well as any supporting
> applications (to generate tests, for example) and tests themselves.
>
> 3. (July 29 – September 16) After confirming that the prototype works well,
> I would start actually integrating it into HAMMER2 filesystem. At this
> point, the task would be gradually adding prototype’s code without any
> conceptual modification into HAMMER2 and testing if it works correctly in
> real-life. A series of testing applications with appropriate system calls
> would be created and the results checked for correctness. Also, the
> appropriate hammer2 utility command to set the compression feature to
> directories would be created and tested.
>
> The deliverables would be the code of implemented feature and application
> tests for it.
>
> 4. (September 16 – End of GSoC 2013) Hopefully, all known bugs would be
> corrected at this point, so I would take the time to do final refinements to
> documentation, to write a detailed description of what was implemented for
> future developers and to finish everything needed to start submitting my
> code to Google.
>
> Generally, my philosophy is to start “as soon as possible”, so if some phase
> is done before the deadline, I would immediately go to the next phase. If
> project goes smoothly and there is some time left, I would be happy to do
> small refinements or bugs fixes on general system or HAMMER2.
>
> During the coding period (June 17 – September 16) I don’t have any
> commitments (like courses or trips) at all, so I should be able to easily
> dedicate 40 hours per week or more to this project. Prior to June 17 I have
> my studies and some exams, but generally it should not prevent me from
> getting familiarized with code and getting the design/algorithm decisions
> done, because my workload is not very big.
>
> I don’t have any past experience with Open Source or generally big projects,
> but I believe that I can accomplish this task since it is generally related
> to C programming and algorithms, and I believe that I have sufficient
> knowledge of those fields from my university courses.
>
> Currently I am getting myself familiarized with Dragonfly 3.2 (for i386)
> running it on Oracle VM Virtual Box 4.2.12, but I’m able to install it on
> physical machine, if necessary. I’m also trying to learn as much as possible
> about BSD in general and particularly Dragonfly.
>
> If you think that my set of skills is more appropriate to some other project
> related to HAMMER2 or Dragonfly, I would be grateful if you pointed at them.
> I also would consider working on HAMMER2 block encryption feature, since it
> is similar in requirements to my current proposal, if there is already a
> strong candidate to implement the block compression feature.
>
> Thank you.
>
> [1] http://www.dragonflybsd.org/docs/developer/gsocprojectspage/
> [2] https://dl.dropboxusercontent.com/u/56646280/zeos.zip
> [3] http://leaf.dragonflybsd.org/mailarchive/users/2012-02/msg00020.html
>
>
> Daniel