Ghana.Ethiopia
From AIMSWiki
| Table of contents |
Table of Contents
Members
- Peter Amoako-Yirenkyi (Ghana)
- Gordon Nana Kwesi Amoako (Ghana)
- Tesfaye Mekonen Yigzawe (Ethiopia)
Results
M O N D A Y
- PROBLEM 1 REPORT
Objective
To download and install the BOINC Client on the Linux(Ubuntu) Platform.
Procedure
- Download the standard BOINC Manager from the BOINC homepage http://boinc.berkeley.edu
- Open the Terminal in Ubuntu
- Grant access rights by entering the following at the prompt chmod 775 ./boinc_5.8.16_i686-pc-linux-gnu.sh
- Install the application by entering ./boinc_5.8.16_i686-pc-linux-gnu.sh at the prompt
- Follow instructions
- Attach to the BOINC project in this case malariacontrol.net ("mcdn" for short)
Results
BOINC client successfully installed by the Group on the Ubuntu Platform.
- PROBLEM 2 REPORT
Objective
Investigate all BOINC projects and find out if some of them grant more BOINC credit than others for the same amount of computing time delivered.
Methodology/Procedure
Extensive research was done on this very topic and the findings are clearly stated in the text that follows.
How the credit system works
The BOINC Credit System is designed to avoid cheating by validating results before granting credit on projects using BOINC as a platform for distributed computing. This ensures users are returning accurate results for both scientific and statistical reasons.
Purposes for a credit system
Online distributed computing relies heavily, if not completely on volunteer computers. For this reason projects, such as SETI@home and other BOINC projects, depend on a complicated balance of long-term users and the cycle of new users and retiring users.
Reasons for participation
- Donate to scientific cause
- Many users wish to advance the specific field of study
- Projects that help fight disease may have an emotional connection for those participating
- Stress test computers
- Teams, credits, and competition
- Many individuals and teams run many computers and have some dedicated specifically to BOINC in hopes of climbing to the top of the world charts
- Personal benefits and recognition
- Projects such as PlanetQuest plan on allowing individuals to name those planets discovered using their computers
- Projects such as BURP plan on allowing users to submit their own 3D models to be rendered using BOINC
Cobblestones
The basis for the BOINC credit system is the cobblestone named after Jeff Cobb of SETI@home. The basis of the system is the concept that 100 cobblestones would be claimed for one day of work on a computer with the following specifications:
- 1,000 double-precision MIPS based on the Whetstone benchmark.
- 1,000 VAX MIPS based on the Dhrystone benchmark.
The actual computational difficulty needed to run a given work unit is the basis for the number of credits that it should be granted. The BOINC system allows for work of any length to be processed and have a user claim identical amounts of credit.
To achieve this BOINC uses benchmarks to measure the speed of a system and in combination with the amount of time it required for a work unit to process can “guess� at the amount of credit it should receive. Since systems have many variables including the amount of RAM, the processor speed, and specific architectures of different motherboards and CPUs, it has been found that wide discrepancies cause there to be differences in the number of credits different computers believe each work unit required to process.
Most projects require a consensus be reached by having multiple hosts return the same work unit. If they all agree then the credit is calculated and all hosts receive the same amount regardless of what they asked for. Each project can use their own policy depending on what they see is best for their specific needs. In general the top and bottom claimed credits are dropped and an average of the remaining is taken.
Total credit
Credits are tracked internally for computers, users, and teams. When a computer processes and returns a work unit it receives no credit. It must first have that work unit validated by the project specific method. Once validated the computer is granted credit, which can be less than, equal to, or greater than what was requested. This amount is immediately added to the computer, user, and team total. If a work unit is returned past the given deadline or is found to be inaccurate it is marked as invalid and the credits granted will be equal to zero. Users and teams commonly determine world rank by comparing the total number of credits accumulated. This highly favors users and teams that have been around for the longest time and makes it extremely difficult for new users to rapidly gain ground even if they are running many computers.
Recent average credit
To calculate the useful amount of work provided by a computer, a special calculation called Recent Average Credit (RAC) is used. This calculation is designed to estimate the number of credits a computer, user, and team will accumulate on an average day. Due to the many variables not taken into account including the inconsistency of host processing, time it takes to validate work units, discrepancies in benchmarks, and possible project down time, the RAC calculation proves to be only a guide. Additionally RAC is independent of computers, users, and teams, meaning they cannot be simply added up. RAC was originally meant to help scientists understand the computational power available to them and to increase competition among users by allowing even new users to quickly move up in rank based on RAC, which should directly reflect how fast work is being processed.
Third party statistics sites
BOINC projects export statistical information in the form of XML files and make it available for anyone to download. Many different third party statistics websites have been developed to track the progress of BOINC projects. The statistics track computers, users, teams, and countries within individual projects and across many projects. Many different sites provide summary graphics, which can be used on web pages that automatically update to contain the statistical information for the specified user or team.
Webpages
- BOINCstats.com—Website (http://www.boincstats.com) by Willy de Zutter
- Boinc.dk—Website (http://www.boinc.dk/index.php?page=statistics) by Janus Kristensen
- BOINC Statistics for the WORLD!—Website (http://www.boincsynergy.com/stats/index.php) by Zain Upton
- Boinc all project stats;Website (http://www.allprojectstats.com) by Markus Tervooren
See also
External links
- Berkeley Open Infrastructure for Network Computing (BOINC) (http://boinc.berkeley.edu/)
- Unofficial optimized BOINC clients for Linux (http://boinc.us.tt)
- Unofficial BOINC Wiki (http://boinc-wiki.ath.cx/index.php?title=Main_Page)
CREDIT: http://en.wikipedia.org/wiki/BOINC_Credit_System
- PROBLEM 3 REPORT
Objective
Find out how the BOINC client manages to control its workload and schedule jobs effectively.
Report
THIS IS HOW IT WORKS
The new version of BOINC postpones reporting to the next time BOINC will contact the servers anyway. This has been built in to avoid too much net load. First, the computation deadline is: the report deadline – (queue size (or connect every X days) + project switch interval (switch projects every X minutes) + 1 day).
The round robin simulator is used to determine if any task would be reported late based on the remaining CPU time left for each task and the resource shares for each project. (A low resource share project will quite often be in deadline trouble).
For each task that is in deadline trouble, start the earliest deadline task that is available in that project up to the limit of CPUs.
If there are no more available CPUs, then do not continue scheduling tasks.
For each remaining CPU, schedule a task from the project with the highest short term debt. Internally track the fact that a task was scheduled from this project and deduct the task scheduling period. (this means if there are two projects with nearly identical STDs, a task will be started from each if there are enough available CPUs.
Once the schedule has been set, the enforcement occurs.
If a project has a task in deadline trouble, start the scheduled task(s) immediately up to the number of tasks in deadline trouble.
For all other CPUs, wait until the task that is being un-scheduled has reached the first checkpoint after it has gotten project switch interval wall time.
If the sum of RAM used by running tasks exceeds the amount of RAM specified, stop tasks until the amount of RAM used is below the specified amount. Note that this can leave CPUs idle.
REFERENCE: [[1] (http://boincfaq.mundayweb.com/index.php?language=1&view=168&sessionID=d99b33595d55e6dc197e442da0648e1a)]
How It Was Achieved
Members of the group did some research online and apparently found how the scheduling works at the BOINC FAQ Pages in one of the Threads. The Report is seen above.
- PROBLEM 4 REPORT
- PROBLEM 5 REPORT
T U E S D A Y
Objective
To Install a BOINC Server
How It Was Done
- Start by installing the PHP and other needed files. This was done by entering the command at the Terminal as follows:
sudo aptitude install php5-cli php5-gd phpmyadmin python-mysqldb m4 make autoconf automake1.9 build-essential libssl-dev libmysql++-dev libtool subversion
- Continue with the installation of the E-Mail Client Mutt
sudo aptitude install mutt ant python-dev python-libxml2
- Add BOINC Users and Groups as follows:
sudo adduser boincadm sudo adduser boincadm admin sudo usermod -G boincadm -a www-data
- Log-In to the boincadm, in this case ours
ssh boincadm@curlew.aims.ac.za
- Set up your mysql database (the root password for mysql database):
mysqladmin -h localhost -u root password ****
- Give the correct permissions to the boincadm user in the mysql database
sudo mysql -h localhost -u root -p
> GRANT ALL ON *.* TO 'boincadm'@'localhost';
> SET PASSWORD FOR 'boincadm'@'localhost'=password('******');
> FLUSH PRIVILEGES;
- Get BOINC source as follows:
sudo aptitude install subversion vim ~/.subversion/servers
- Add in the [global] section:
http-proxy-host = proxy.aims.ac.za http-proxy-port = 3128
- Now use subversion to fetch the source, compile and test it.
svn co http://shrike.aims.ac.za/svn/trunk/ boinc_trunk cd ~/boinc_trunk ./_autosetup ./configure --disable-client make sudo make install cd /usr/local/src && sudo ln -s /home/boincadm/boinc_trunk boinc
- We connected to test our server by going to the URL
[[2] (http://curlew.aims.ac.za)]
Goto to web server root directory /var/www/ and create a php test page
sudo vim /var/www/phptest.php
with these statements
<?php echo "hello world!"; phpinfo(); ?>
- Test page by pointing your browser to [[3] (http://curlew.aims.ac.za/phptest.php)]
CREATING A PROJECT ON THE BOINC SERVER
In this project we joined as existing project. We type the following at the Terminal
cd ~/boinc_trunk/tools ./make_project --test_app wagtail
The output below is seen:
Creating project 'wagtail' (short name 'wagtail'):
PROJECT_ROOT = /home/boincadm/projects/wagtail/
URL_BASE = http://wagtail.aims.ac.za/
HTML_USER_URL = http://wagtail.aims.ac.za/wagtail/
HTML_OPS_URL = http://wagtail.aims.ac.za/wagtail_ops/
CGI_URL = http://wagtail.aims.ac.za/wagtail_cgi/
KEY_DIR = /home/boincadm/projects/wagtail/keys/
DB_NAME = wagtail
DB_HOST =
- Now copy the web server configuration file to the web server configuration directory as follows:
cp /home/boincadm/projects/wagtail/wagtail.httpd.conf /etc/apache2/conf.d
and restart the webserver as follows:
sudo /etc/init.d/apache2 restart
- We add a cron as follows:
boincadm@curlew:~$ crontab -e
Add a job as follows:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/boincadm/projects/wagtail/bin/start --cron
- Register the application with the current project for AIMS Africa@Home course:
cd ~/projects/wagtail bin/xadd bin/update_versions
The defult homepage had a bug that was fixed as follows:
vim html/user/index.php
The line that read xml.gif was changed to img/xml.gif
- We edited the following to suit the our project
vim html/project/project.inc
define("PROJECT", "Ghana.Ethiopia BOINC Project");
define("COPYRIGHT_HOLDER", "Ghana.Ethiopia@AIMS");
- Make it possible to make volunteers to attach to new project by doing the following:
vim config.xml
Change
change <disable_account_creation> from 0 to 1
- Line 60 of the project's user.inc file (PROJECTHOME/html/inc/user.inc) was fixed as follows:
function get_other_projects($user) {
'''return $user;'''
$cpid = md5($user->cross_project_id . $user->email_addr);
$url = "http://boinc.netsoft-online.com/get_user.php?cpid=$cpid";
$f = fopen($url, "r");
if (!$f) {
return $user;
}
$u = parse_user($f, $user);
fclose($f);
return $u;
}
- Project homepage was customized to suit us
Our project homepage is then [[4] (http://curlew.aims.ac.za/wagtail/)]
BEGINING USER OPERATIONS
- We connected our client to our project [[5] (http://curlew.aims.ac.za/wagtail/)] using the BOINC manager.
Screenshots are provided below:
 |
 |
The project started working succesfully and EUREKA we had our first job done at time Unix Time 1184687388 equivalent to17:49:48 verified by ,Christian Ulrik Sottrop, Volunteer Tutor for the AIMS Africa@Home Workshop.
W E D N E S D A Y
E X E R C I S E ONE REPORT
Objective
The malariacontrol.net contains calls to the BOINC API in several places. This is a good place to study the interaction between the science application and the core client.
How It was Achieved
The mcdn source was downloaded from : [6] (http://shrike.aims.ac.za/mcdn.tar.gz) and unpacked as follows:
$mcdn/prerequisites.sh $cp -r boinc_trunk mcdn/boinc
E X E R C I S E TWO REPORT
Objective
Understand the function of the science application when run in standalone mode.
How It was Achieved
- The science application had to be built but some prerequisites must be met they are:
**Obtain the Intel Fortran compiler from the source [[7] (http://shrike.aims.ac.za/l_fc_c_10.0.025.tar.gz)]
Note: Register at [[8] (http://www.intel.com/cd/software/products/asmo-na/eng/compilers/flin/282048.htm)] to obtain a license
- Unpack the compiler then cd to the installer directory.
$sudo install.sh
- Append the following two lines to your .bashrc
source /opt/intel/fc/10.0.025/bin/ifortvars.sh source /opt/intel/idb/10.0.025/bin/idbvars.sh
- Hide some .so files to force static linking (workaround)
$sudo mkdir /opt/intel/fc/10.0.025/lib/bak $sudo mv /opt/intel/fc/10.0.025/lib/*so /opt/intel/fc/10.0.025/lib/bak
E X E R C I S E THREE REPORT
Objective
Understand the function of the malariacontrol.net server components and setup a malariacontrol.net test server.
How It was Achieved
Begin by handling prerequisites(installation of needed files) as follows:
- Create a project root directory and a malariacontrol project
$sudo mkdir -p /malaria/malaria $sudo chown -R boincadm:boincadm /malaria
$tools/make_project --project_root /malaria/malaria --delete_prev_inst --drop_db_first malariacontrol 'malaria control.net test'
- Build the daemons and tasks in mcdn/mcdnServer.
- Add assimilator,generator,parameterizer,validator to the PROJECT_ROOT/bin directory eg. as below in our case:
boincadm@curlew:/malaria/malaria$ cp validator parameterizer generator assimilator /malaria/malaria/bin/
- Update the PROJECT_ROOT/config.xml:
<task>
<cmd>
generator -db_name malariaModel -template_name ../templates/generator_template.xml -d 3
</cmd>
<period>
1 min
</period>
</task>
<task>
<cmd>
parameterizer -app malariacontrol -db_name malariaModel -min_results_ready 9 -template_name ../templates/generator_template.xml -d 3
</cmd>
<period>
1 min
</period>
</task>
<daemon>
<cmd>
assimilator -d 2 -app malariacontrol -db_name malariaModel -template_name ../templates/generator_template.xml -min_results_ready 14
</cmd>
</daemon>
<daemon>
<cmd>
validator -d 3 -app malariacontrol
</cmd>
</daemon>
- Update project.xml
<app>
<name>malariacontrol</name>
<user_friendly_name>malcont</user_friendly_name>
</app>
- Run
$bin/xadd
Create the app dir and add the science application
$mkdir -p apps/malariacontrol/malariacontrol_5.01_i686-pc-linux-gnu
- Now it's time to copy and sign the science application.
- Compile the malaria science app in mcdn/malariaModel/cpp/malaria to the application dir. Note: the application name must be the same as the directory name.
cp /home/boincadm/mcdn/malariaModel/test/sandbox/malaria /malaria/malaria/apps/malariacontrol/malariacontrol_5.01_i686-pc-linux-gnu/malariacontrol_5.01_i686-pc-linux-gnu
crypt_prog -sign /malaria/malaria/apps/malariacontrol/malariacontrol_5.01_i686-pc-linux-gnu/malariacontrol_5.01_i686-pc-linux-gnu /malaria/malaria/keys/code_sign_private > /malaria/malaria/apps/malariacontrol/malariacontrol_5.01_i686-pc-linux-gnu/malariacontrol_5.01_i686-pc-linux-gnu.sig
crypt_prog -verify /malaria/malaria/apps/malariacontrol/malariacontrol_5.01_i686-pc-linux-gnu/malariacontrol_5.01_i686-pc-linux-gnu /malaria/malaria/apps/malariacontrol/malariacontrol_5.01_i686-pc-linux-gnu/malariacontrol_5.01_i686-pc-linux-gnu.sig /malaria/malaria/keys/code_sign_public
- Update your application version
$bin/update_versions
- Initialize the backend db: Create a database named malariaModel, then
$mysql -p malariaModel<~/mcdnServer/templates/mmDBInit.sql
- Copy template files
cp /home/boincadm/mcdn/mcdnServer/templates/*xml /malaria/malaria/templates/
- Copy datafiles:
mkdir /malaria/malaria/files cp /home/boincadm/mcdn/mcdnServer/data/densities.csv /malaria/malaria/files/ cp /home/boincadm/mcdn/mcdnServer/data/lossWeightVector.txt /malaria/malaria/tmp_HOSTNAME cp /home/boincadm/mcdn/mcdnServer/data/fieldData.txt /malaria/malaria/tmp_HOSTNAME
- BOINC server restart
bin/stop bin/start
- Adding run1
$mysql -p malariaModel <~/mcdnServer/data/run1.sql
- Make web directories and scheduling as done for the Wednesday Exercise
- Work on the cron
*/5 * * * * /malaria/malaria/bin/start --cron
Below is the first screenshot of workdone:
 |
