Rant of the Week (ROTW): Facebook

Facebook has morphed yet again. But this time, instead of just changing the interface, it actually changes your profile.  My profile went from being something relatively custom-fit to me, to being a set of links to pages that are tangentially related to me.

Of course this makes sense from a business perspective, it makes it a lot easier to put together sets of interests to crunch numbers for advertising purposes.  However, I feel the social value of my profile goes down, because anything that is not expressible as a "page" is no longer allowed outside of the Bio / Quotations field.

My knee-jerk reaction: delete everything that links my profile to other pages.  If Facebook wants to make advertising $$ off of me, they'll have to infer things in a less obvious and heavy-handed way.  Honestly, I'd rather have a stripped-down profile than one with links to pages that I don't really feel strongly about.  It would not be so bad if it were an opt-in process, for example if I could mix linked content with unlinked content.  However, being forced to use one mode of expression, i.e. linked pages, is unacceptable to me.



I won't go so far as to quit Facebook, it still has some value.  For example, sharing thoughts with friends through the status updates, sharing photos and photo albums, and connecting/staying in touch with old friends.  But for these things I do not need to be part of an inclusive web of linked pages in order to share information about myself.

It's bad enough when I don't know what Facebook is doing with my information. (I assume selling as much of it as possible without getting too much bad publicity.) However, dictating what I can and cannot say about myself, especially for mundane non-inflammatory things such as what movies I like, is draconian Stalinism that has no place in what should be an open environment for the dissemination of opinions and ideas.

Read More

A week in M5

I spent the last week or so playing around with a relatively new simulator called M5.  It is a very nice simulator, especially for architectural research.  My interest in it is to use it for doing both operating system and architecture research.  This type of research is difficult to do with most simulators, because fully functional simulators rarely provide accurate timing to evaluate overheads, while timing simulators rarely provide sufficient functionality to run an operating system.  What is needed is a full system simulator that provides fidelity for both functionality and timing.

In the past our research group has used SimpleScalar for architectural research.  However, that project is obsolete and the simulator is slowly becoming obsolete with respect to real-world systems.  Also, it does not provide full system simulation, although a set of patches by Jack Whitham merges RTEMS and SimpleScalar to provide full system simulation for RTEMS.  I tried to use Jack's patches, but I could not quite get everything working.  When I sent him an e-mail to ask about it, he suggested that I should look at M5 as a more realistic simulator instead.  So I did.

M5 provides two types of simulators: Full System (FS) and Syscall Emulation (SE).


FS simulation is what I need, although it might be possible to layer experiments on an SE simulator using pthreads or something similar.  I have had trouble getting the UltraSPARC T1 SE (SPARC_SE) simulator to work with my RTEMS Sparc64 Port, although it might work with a little bit more massaging. However, I'm not sure it is sufficient for the type of research I want to do, particularly with RTEMS.  The main problem I see is that interrupts won't be delivered to the target (RTEMS), because the interrupt sources are not actually simulated.  This means there will be no timers and no pre-emptive multitasking, i.e. a very limited feature set.

According to the M5 wiki, the FS Simulator currently supports only two ISA's: DEC Alpha with up to 4 cores (64 cores on an alpha-derived CPU that has no real-world analog) and UltraSPARC T1 with 1 core.  These are referred to as ALPHA_FS and SPARC_FS respectively.

M5 also provides a variety of CPU models, with varying capabilities.

• AtomicSimpleCPU model is basic functionality only.
• TimingSimpleCPU provides CPI=1 with stalls on loads.
• O3CPU provides a time-accurate out-of-order pipeline model
• InOrderCPU provides a time-accurate in-order pipeline model
• Checker is used to provide functional correctness.

As far as I can tell, SPARC_FS so far officially only works on AtomicSimpleCPU. It might work with TimingSimpleCPU, but that would be as good as I can do with Simics Niagara and the Ruby module from Wisconsin GEMS.  Unfortunately, I haven't actually been able to get the TimingSimpleCPU to work with SPARC_FS.  This means SPARC_FS is not very useful for doing architecture research.  However, as an open source simulator, it is a compelling project for helping to ensure that the Sparc64 RTEMS port can be tested.

ALPHA_FS appears to be capable of running on any of the CPU models, except for InOrderCPU.  According to the mailing list, full system is not implemented for any of the InOrder platforms.  However, RTEMS does not support the DEC Alpha ISA at all, and I'm not aware of any interest in it as a port.

Booting OpenSolaris on M5 SPARC_FS
Follow the instructions on the M5 wiki for getting started.  The examples given are for the Alpha targets.  You will also want to compile and install the m5term application.

To build the sparc full system targets, use:

$ scons build/SPARC_FS/m5.debug 
$ scons build/SPARC_FS/m5.opt
$ scons build/SPARC_FS/m5.prof
$ scons build/SPARC_FS/m5.fast

Download and extract the OpenSparc OpenSparc architecture and performance modeling tools. Copy *.bin and nvram1 from OpenSPARCT1_Arch.1.5/S10image/ to the /dist/m5/system/binaries/ directory.  Also copy disk.s10hw2 from the S10image/ directory to the /dist/m5/system/disks/ directory.  Rename reset.bin, q.bin, and openboot.bin to reset_new.bin, q_new.bin, and openboot_new.bin, which are the binaries expected by the m5 SPARC_FS scripts.

In a terminal window, start the simulator with:

$ build/SPARC_FS/m5.debug -d /tmp/output configs/example/fs.py

In another terminal, connect to the simulator with:

$ m5term localhost 3457

 You should eventually see this in your m5term window:

==== m5 slave terminal: Terminal 0 ====
Sun Fire T2000, No Keyboard
Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
OpenBoot 4.20.0, 256 MB memory available, Serial #1122867.
[mo23723 obp4.20.0 #0]
Ethernet address 0:80:3:de:ad:3, Host ID: 80112233.



ok

 The "ok" prompt is the OpenBoot prompt.  Just type boot and press enter, and the OpenSparc Solaris image will start to boot.

Booting RTEMS on M5 SPARC_FS
The sparc64 sun4v BSP will boot on the SPARC_FS full system simulator of M5.  The first step is to bundle RTEMS executables and SILO on to a bootable ISO9660 filesystem.  We have some scripts to help create the bootable disk, and use the same approach for booting RTEMS on Simics Niagara.  Next change the ${m5}/configs/common/FSConfig.py file and replace disk('disk.s10hw2') with disk('image.iso'), where image.iso is the ISO9660 filesystem image built for booting RTEMS.  M5 will look in /dist/m5/system/disks for the image.iso file, so link image.iso to the /dist/m5/system/disks/image.iso location.

I have so far verified that Hello World (hello) and the ticker will boot and execute to completion.  However, there is no facility built into M5 to set OpenBoot parameters, in particular I cannot set auto-boot = true; I would need to create an nvram1 with this setting.  I could probably write a script to drive the interactive portions of the m5term dialog, maybe even with a configuration checkpoint at the SILO boot prompt.

There are a few other tasks left that would be nice to resolve.  First, the simulator does not exit when the RTEMS application finishes.  This could be resolved by a script to detect the end of execution, which is what is done now with Simics, or by adding some instructions to cause the application to terminate.  Second, it might be nice to get RTEMS to run on SPARC_SE, although the functionality that would be provided on this platform is questionable.  Third, getting things to work on the InOrderCPU model would be ideal, although that functionality is not currently supported in m5 for SPARC_FS.  Fourth, I was not able to hook up the remote debugger (gdb) without getting some strange errors that are probably related to the flavor of gdb and the ABI assumed by M5's UART.  This last task was the result of a problem I faced trying to get SPARC_SE to work.

Overall, I found it was easy to compile and build the M5 simulator.  I also got a chance to go into the source, but I did not go very far so I will withhold judgement there.  I am excited that the basic AtomicSimpleCPU SPARC_FS is able to boot and run RTEMS, which will help get the Sparc64 RTEMS Port accepted, included, and maintained in the RTEMS code base.

Read More

vim Productivity

I use vim.  Not because I think it is inherently superior to emacs or any other full-featured editor/IDE, but because I learned it first.  A fellow vim user and friend of mine tried emacs out for about a semester+ and his experience was that they are roughly equivalent after learning how to use each, so I stick with vim.  The only real difference between the two is the command syntax

Through working on various projects, I have gathered a few tricks, tips, and features to improve my productivity while using vim.  The rest of this post is a semi-tutorial and repository of my vim usage.

Built-ins:

• .vimrc
• Modes
• Cut-Paste
• Find-Replace
• Jumping Around 
• Split
• Auto-Format 
• Shell commands

Plug-ins:

• Latex-suite
• Cscope
• changelog

.vimrc
One of the most useful things to learn (or at least do) with vim is to get a good .vimrc.  The .vimrc is read from your home directory when you open vim, and it can be used to override and set many options to customize your vim environment.  Look around online for vimrc options and settings, here is a sample vimrc that is close to what I use:

" Use Vim settings, rather then Vi settings (much better!).
" This must be first, because it changes other options as a side effect.
set nocompatible

" allow backspacing over everything in insert mode
set backspace=indent,eol,start

" change spacing for tabbing and autoindent
set tabstop=2 "number of columns to display in a tab
set shiftwidth=2 "how many columns to indent with autoindent
set expandtab " inserts spaces instead of tab
set autoindent " always set autoindenting on
set smartindent

set nobackup " do not keep a backup file, use versions instead
set history=50 " keep 50 lines of command line history
set ruler " show the cursor position all the time
set showcmd " display incomplete commands
set incsearch " do incremental searching

" Don't use Ex mode, use Q for formatting
map Q gq

" Switch syntax highlighting on, when the terminal has colors
if &t_Co > 2 || has("gui_running")
 syntax on
"set background=dark " use with dark terminal windows.
"set hlsearch "highlight the last used search pattern
endif

" Only do this part when compiled with support for autocommands.
if has("autocmd")

" Enable file type detection.
" Use the default filetype settings, so that mail gets 'tw' set to 72,
" 'cindent' is on in C files, etc.
" Also load indent files, to automatically do language-dependent indenting.
filetype plugin indent on

" For all text files set 'textwidth' to 80 characters.
autocmd FileType text setlocal textwidth=80
endif

set backupcopy=auto,breakhardlink

A lot of the options in the .vimrc help with formatting.  My options provide for tabs to be two white space characters.  This also turns on syntax highlighting, indentation, and breaking hard links, which I mentioned in my previous post about versioning work.

Modes
Vim has a number of modes that change how you interact with the editor. The modes that I use are normal, insert, command-line, and visual.  There are some other modes, but I avoid them.  Normal mode is the mode you start in, and is mainly useful for navigating through text and issuing simple commands.  Insert mode is entered by pressing i from normal mode, and is used for editing text. Command-line mode is entered by pressing the colon key, :, while in normal mode. You need to use command-line mode to save and to quit vim, for example, while in normal mode pressing :wq will save and quit, :q! will quit without saving changes; the help menu is also accessed using :help. Visual mode is entered by pressing v from normal mode, and I will explain some of its uses in the next paragraph.

To enter visual mode, hit the esc key (to enter normal mode) and then press v.  Then you can move your cursor around with the arrow keys to highlight text. Visual mode is useful for making controlled changes within your text.  To get back to editing, hit esc and then press i to enter insert mode.  With visual mode, you can use vim commands to operate on the text that you highlight. Simply highlight the region of text you want to search and replace in, and then hit :, which will bring up a command that is automatically scoped to the range you have highlighted, which looks like this :'<,'> . Some of the remaining sections will discuss ways to use visual mode for a more advanced vim experience.

Cut-Paste 
Probably one of the more important features of an editor is the ability to move chunks of text around efficiently.  Cutting and pasting is easy enough in vim.  First let's see how to copy text. To copy a line of text, enter normal mode and press yy. This will "yank" the current line of text.  To paste what you yanked, enter normal mode and press p, which will paste the yanked line wherever the cursor is located.

You can also cut a line of text, enter normal mode and press dd, which deletes the line and saves it in vim's clipboard.  You can also yank or delete two lines, enter normal mode and press either:

• d [down arrow OR up arrow]
• y [down arrow OR up arrow]

These two commands will cut or copy (respectively) the current line and the next or previous (respectively) line.  Then you can paste the two lines anywhere you like. It is also possible to cut/copy multiple lines.  Do this by entering command mode Typically when I need to get more than two lines I always use visual mode.

Enter visual mode and then highlight the text you want to cut/copy.  While still in visual mode, press d or y to cut or copy the highlighted text -- you will be returned to normal mode, and can paste your text where you need.  This method allows you to control precisely which text to cut or copy, you can select parts of lines and even just part of a single line, or a single word for example.  The only real limitation is that you can only highlight contiguous regions, at least I haven't learned how to cut multiple selected regions at once.  I also use this method to just delete text that is no longer needed by highlight and pressing d, without ever pasting the clipboard buffer contents anywhere.

One limitation to this method of copy and paste is that the vim clipboard buffer only works in a single instance of vim.  This means with the above method you cannot copy/paste between vim instances, or between any other program and vim.  Also, with auto-indent and other features enabled, pasting with the terminal window (which copies/pastes the system clipboard) or mouse buffer can lead to some very messy formatting issues. I recently learned how to get around this limitation, and can now copy and paste nicely between vim and the system or mouse clipboards.

How to copy and paste between vim instances
To copy a line of text from vim into the system or mouse clipboards, enter normal mode and press "+yy or "*yy respectively.

You can also use visual mode to copy the highlighted text to the system and mouse clipboards. In visual mode, highlight the text you want to copy, and press "+y or "*y.  Cutting text works identically, but with d instead of y.  You can also paste from either clipboard from normal mode with "+p or "*p.

These two paste commands work for text copied in other applications or for the text currently highlighted by the mouse, and using the p command instead of the terminal (ctrl+shift+v) or mouse paste (middle mouse click) will prevent vim from attempting to automatically format the pasted text.

I have had some trouble with this in Fedora, but it works well with Ubuntu.

Find-Replace
There's almost always a need to do a little search, and maybe even some replace.  Find in vim is easy, just hit / and then type what you want to find.  To do find and replace, you use the : to enter command mode (first press esc if you are in insert mode; commands can be entered directly in visual mode).  From normal mode, pressing :s/find/replace will replace the first instance of find with replace in the current line.  To replace every instance of find with replace in the current line, you need to add a /g to the end :s/find/replace/g

Usually what you want to do is to replace every instance in the entire file.  Do this by adding a range specifier before the s, for example :%s/find/replace/g

You can also do find replace over a range, for example between lines 1 and 100 you would enter :1,100s/find/replace/g

This is really easy to do if you use visual mode. As explained before, simply highlight a region and press :, then s/find/replace/g.

Jumping Around
Despite not supporting a mouse, after learning a few tricks it is exceedingly simple to navigate through text. To jump to a specific line number, simply enter the line number in command mode, for example :42 will jump to line number 42. You can also jump to relative line numbers, for example :-42 or :+42 will jump back or forward 42 lines respectively.  And if you don't know an exact line number, you can also jump to an approximate line number, for example pressing 42% in normal mode will place your cursor past 42% of the file.

There are also commands to jump to specific places in a file.  For example, entering gg in normal mode will jump to the start of the file, and entering G will jump to the end of the file. There are also useful commands to move within a single line: pressing ^ will move your cursor to the start (first non-whitespace character) of the current line, and pressing $ will move your cursor to the end of the line.  There are a lot of other ways to efficiently move around, but these are the commands I use most often.

Split
Quite often I work on one file and want to reference another file. Using the split command lets me open multiple files in the same terminal and same instance of vim.  For example, entering :split foo.c will open the file foo.c (assuming it is in the same directory in which vim started) in a split portion of the current window.  You can open any file by passing relative or absolute paths to the split command.

Window commands begin by first pressing ctrl-w and another letter.  To cycle between open windows, use ctrl-w w. You can control the window to move to, with horizontal splits use ctrl-w j (or up arrow) or ctrl-w k (or down arrow) to move to the window above or below your current window.  :vsplit will create a vertical split.  I almost always use regular split with a file name and cycle through windows with ctrl-w w.

I have also previously learned how to use tab pages, which lets you open multiple files in tabs, where each file takes up a whole window. To open a file foo.c in a new tab, use :tabedit foo.c, which works like split except it creates a tab.  You can also open the same file as you are currently editing with :tab split. To cycle through the tabbed windows use gt in normal mode, or gT to go to the previous tab.

Auto-Format
Before I learned to use "+p and "*p to paste, I would often paste code and it would be really badly formatted.  With the autoindent options set in the .vimrc, I can easily reformat the text.  A single line can be formatted by pressing == in normal mode, but that is usually not sufficient.  I almost always use visual mode to highlight the text I want to format, and then press =.  You can also reformat the entire file, by going to the start of the file with gg then pressing = followed by G.

Shell commands 
From the vim command-line, you can also issue arbitrary commands to the underlying shell using the ! operator.  For example, :!pwd will print the current working directory and prompt to return to editing.  You can also pass data to another program like using a pipe, for example you can highlight a range and then sort the lines by using :'<,'>!sort; note that the output of the external program will replace the input.  More complicated expressions can also be constructed, although I don't use this feature of vim very often.

Latex-suite
One of the first plug-ins that I ever tried is the latex-suite; Ubuntu provides a package called vim-latexsuite.  Although it has a slight learning curve, it is pretty easy to pick up and use.  Some of its nicer features that I use are templates, citation completion, and folding for latex.  One annoyance with the latex-suite is the macro expansion, which occasionally expands text that I don't want expanded.  For this, I can set b:Imap_FreezeImap to 1 to pause it for the current window, or set g:Imap_FreezeImap to 1 in the .vimrc to turn it off completely.

Templates are LaTeX files that have placeholders for customizing the file with your data. I typically just use the templates to start new projects and replace all of the placeholders with blanks, but there is some support for moving through the placeholders.  Citation completion makes it easy to insert citations by simply typing \cite{ and then starting the citation completion.  Folding is a way to hide text, which can make it quicker to scan through a file to find a section that you want to work on.  Folds are opened with the command zo and closed with zc.  Latex suite takes care of creating the folds for you.

I tend to use gvim with latex-suite, since the GUI options are nice and I don't have to remember the extra key bindings.

Cscope
I haven't used Cscope much, but it is a nice tool for moving through a source code tree quickly.  Cscope is especially useful for finding function or symbol definitions and uses.  I learned how to use it with vim from an online tutorial.  One of the keys to successfully using Cscope is to generate the database it uses for searching for symbols.  What you provide to Cscope is a file containing a list of all the files for it to include in the database.  I use the find command to generate this list.  For example, to generate the Cscope database for RTEMS I use the following commands (with $r equal to the top level of the rtems source directory, and $R is the parent of $r):

find $r/c $r/cpukit /$r/testsuites -regex '^.*\.[chsS]' -print > $R/rtemssources
cd $R
rm -f cscope.out
cscope -bki rtemssources

This will create the cscope.out database file.  If you have the cscope_maps.vim file in a good location, then the only other thing to do is to show vim where to find the cscope.out file.  I typically use Cscope to find files and to search for symbol and function definitions. Vim's command-line mode can be used to issue Cscope commands, for example :cs find f foo.c will search for the file foo.c and either open it if it is unique in the database or give you a menu of files named foo.c to choose from.  There are also key bindings in the cscope_maps.vim file that define some useful shortcuts.  The one I use most often is to put my cursor on a symbol (e.g. a struct type or function name) and then hit ctrl-\ s or ctrl-@ s (control + spacebar + s), which will display a list of all references to the symbol.  I can select the one I want by typing in a number included with the list, and it will open in the same window (if I used ctrl-\) or in a horizontal split (if I used ctrl-spacebar).  There are a lot of other ways to use Cscope -- the tutorial gives some, and the cscope_maps.vim file lists a few more.

changelog
One of the projects that I have been working on a lot recently is RTEMS.  The RTEMS project uses Changelogs to track modifications to the source files.  These files have a simple but strict format, and manually creating ChangeLog entries is tedious.  Vim has plugin support for generating ChangeLog entries.  Simply add this to your .vimrc:

" Changelog
runtime ftplugin/changelog.vim
let g:changelog_username = "Your Name "

I also found that the default behavior for the ChangeLog plugin is to merge all entries by the same user on the same day together in a single ChangeLog.  I prefer to have separate entries for each time that I make a new entry, this way I can more easily track back the order of things that I change, and I also usually make an entry and then commit to my version control system.  This means I need to change the plugin behavior.  I did this by copying the default changelog.vim from the usual ftplugin directory, on my computer this is /usr/share/vim/vim72/ftplugin/changelog.vim, and I put the copy in ~/.vim/ftplugin/changelog.vim.  Then I made some manual changes, but here is a diff:

--- /usr/share/vim/vim72/ftplugin/changelog.vim    2010-05-02 16:23:09.000000000 -0400
+++ /home/gedare/.vim/ftplugin/changelog.vim    2010-05-02 19:16:42.000000000 -0400
@@ -178,21 +178,21 @@
     call cursor(1, 1)
     " Look for an entry for today by our user.
     let date = strftime(g:changelog_dateformat)
-    let search = s:substitute_items(g:changelog_date_entry_search, date,
-                                  \ g:changelog_username, prefix)
-    if search(search) > 0
+ "   let search = s:substitute_items(g:changelog_date_entry_search, date,
+ "                                 \ g:changelog_username, prefix)
+ "   if search(search) > 0
       " Ok, now we look for the end of the date entry, and add an entry.
-      call cursor(nextnonblank(line('.') + 1), 1)
-      if search(g:changelog_date_end_entry_search, 'W') > 0
-    let p = (line('.') == line('$')) ? line('.') : line('.') - 1
-      else
-        let p = line('.')
-      endif
-      let ls = split(s:substitute_items(g:changelog_new_entry_format, '', '', prefix),
-                   \ '\n')
-      call append(p, ls)
-      call cursor(p + 1, 1)
-    else
+ "     call cursor(nextnonblank(line('.') + 1), 1)
+ "     if search(g:changelog_date_end_entry_search, 'W') > 0
+"    let p = (line('.') == line('$')) ? line('.') : line('.') - 1
+    "  else
+    "    let p = line('.')
+    "  endif
+    "  let ls = split(s:substitute_items(g:changelog_new_entry_format, '', '', prefix),
+    "               \ '\n')
+    "  call append(p, ls)
+    "  call cursor(p + 1, 1)
+    "else
       " Flag for removing empty lines at end of new ChangeLogs.
       let remove_empty = line('$') == 1

@@ -214,7 +214,7 @@

       " Reposition cursor once we're done.
       call cursor(1, 1)
-    endif
+"    endif

     call s:position_cursor()

 Now whenever I make a change, I simply type \o in normal mode, which splits a window and opens the nearest ChangeLog file (searching up the directory tree) with a new entry for me to fill just the modifications I made.  It automatically populates my name, email (from the .vimrc variable setting), the date, and the file name that I was editing.  Then I can save and close the ChangeLog, and either resume editing or close the modified file if I'm done with my edits.  Typically I only make ChangeLog entries right before closing a file, because then I will also be committing my changes to the version control system.

Read More