Code Crawler Final Report

Version 1.0

Code Crawler Final Report

Version: 1.0 Date: 10/26/2013

Revision History
Date 5/3/2005 Version 1.0 Initial entry Description Nikolay Author

CodeCrawler, 2004

Page 2 of 10

Code Crawler Final Report

Version: 1.0 Date: 10/26/2013

Table of Contents
1. Project 2. Team 3. Roles 4. Distribution of Work 5. Develop Software Interactively 6. Manage Requirements 7. Visually Model Software 8. Continuously Verify Software 9. Control Changes to Software 10. Functionality 11. Metrics 12. Transition 12.1 Install Wizard 12.2 Web Site 12.3 Press Release 13. Documentation 13.1 Source Code 13.2 RUP Artifacts 13.3 Complete Document List 4 4 4 5 6 6 6 7 7 7 7 7 7 8 8 8 8 8 9

CodeCrawler, 2004

Page 3 of 10

Code Crawler Final Report

Version: 1.0 Date: 10/26/2013

Software Requirements Specification

1. Project
The CodeCrawler project was one of a few ideas discussed by group 49 (Kinis, Jebeh, and Nikolay) for CS427. After initial considerations, it was chosen and a vision document was produced describing the proposed product, target audience, features, and competitive products. CodeCrawler survived the initial elimination of project proposals, and attracted five more developers Adam, Edward, David, Mohan, and Martin who could contribute to the project with their Java, testing, design skills and other experiences. With the beginning of CS428 we had to fill in several team member vacancies in order to replace Edward and Adam who did not take the class. Omar and Girish joined the team, but Girish later dropped the class. The final CodeCrawler team consists of only seven members which posed some difficulties which resulted in scaling back some of the proposed functionality for the system. CodeCrawler follows the Rational Unified Process as its development process. All documentation, source code, mailing lists, and bug reports for the CodeCrawler project can be found at http://codecrawler.tigris.org

2.

Team
Kinis Meyer (kmeyer) Jebeh Kawah (kawah) Nikolay Kojuharov (kojuharo) contact person Edward Fancher (fancher2) Martin Mailloux (mailloux) Mohan Kishore (mkishore) David Peerenboom (peerenbo) Adam Wisely (awisely) Omar Ghaffar (ghaffar2)

3.

Roles
Nikolay Nikolay All

Project Manager System Analyst Use-Case Specifier

User-Interface Designer Jabeh Architect Designer Implementer Test Manager Tester Martin All All Omar All

Configuration Manager Mohan CodeCrawler, 2004 Page 4 of 10

Code Crawler Final Report Requirements Reviewer All Architecture Reviewer Design Reviewer Code Reviewer Technical Writer All All All Kinis, David

Version: 1.0 Date: 10/26/2013

4.

Distribution of Work

CONTRIBUTIONS

Adam Wisely and Jebeh Kawah specified two use cases Add Support for New Language and View
Source Code. They also produced several UML diagrams, including a state diagram for the user interface, and created the projects Glossary document. Adam reviewed all project documents and sent comments and suggestions to the authors. Jebeh participated in writing the original project proposal. She identified the initial technical risks for the project. In the prototype, she worked on the layout of the web page displaying formatted source code.

Kinis Meyer and David Peerenboom specified two use cases Create New Codebank and
Changing Existing Codebank, which later became the Change Settings use case. They collected requirements derived from the different use cases and produced and maintained the Software Requirements Specification document. UML diagrams on which they worked include class and collaboration (communication) diagrams for creating a new project codebank and finite state diagram for index. Kinis wrote the original Project Plan and contributed to the CodeCrawler Vision Statement. She also reviewed and provided important feedback to other project documents. David generated the original SRS document, continued its development, collaborating with Kinis.

Edward Fancher and Mohan Kishore specified two use cases Search and View Source Code
and designed related class, sequence, collaboration diagrams. They also designed the state and data flow diagrams for Indexing use case. They collaborated on the design for search query parsing and search results ranking algorithm. Edward, as the test manager, designed tests and wrote the projects Test Plan document. He implemented unit tests for classes in the prototype. Mohan Kishore, as the configuration management specialist, designed the overall strategy for deployment and build integration. He wrote and maintained the Configuration Management Plan, Deployment Plan, and Integration Build Plan documents. He worked on source code for loading the configuration, displaying search results, and displaying formatted results.

Martin Mailloux and Nikolay Kojuharov specified two use cases Build Search Index and
Perform Search. They produced designs for the Indexing and Codebank packages. Martin, as the architect, developed the overall product architecture, and implemented the core architectural CodeCrawler, 2004 Page 5 of 10

Code Crawler Final Report

Version: 1.0 Date: 10/26/2013

prototype. In particular, he wrote the search and indexing functionality as well as the basic mechanism for modelview-controller interaction. Nikolay, as the project manager, oversaw the progress of the project, scheduled tasks, and managed risks. He implemented interfaces and integration of third party C/C++ components with Java code analysis and code formatting. He designed the user interface prototypes, maintained the Use Case Model document, and added use cases for Abort Indexing and Swap Search Index.

5.

Develop Software Interactively

The CodeCrawler Project Plan outlined nine development phases over the two semesters. Inception Fall 2004 Elaboration I, II, III Fall 2004 Construction I, II, III, IV Spring 2005 Transition Spring 2005 At the beginning of CS428 CodeCrawler had an architectural prototype. Since this prototype was planned as a throw-away one we started development from scratch but kept some of the interfaces (e.g. Codebank, Formatter, etc.). At the beginning of each iteration we distributed development tasks between team members and at the end of each iteration we reviewed the implemented functionality and verified with existing tests. We also collected metrics and bug reports in order to track the progress of the project and discover risks. Throughout the development we refactored the code and grew the test base to ensure product quality. During development some weaknesses of the design were discovered so the design and requirements documents had to be modified. At the end of the second construction iteration we released a Beta version of the product which unfortunately didnt have all functionality in it. As a result we had to slate a few planned features for future releases.

6.

Manage Requirements

Development mostly followed requirements specified in the Software Requirements Specification from last semester. However, in many cases those requirements were either incomplete, too demanding, or could not be met at all in which cases the requirement change requests were logged in the issue tracking system and meetings were convened to resolve the issue and derive new requirements. VERIFY SRS and Design are up to date

7.

Visually Model Software

Last semester we extensively used visual aids and diagrams in specifying and designing the project. Class, sequence, communication, and state UML diagrams as well as data flow diagrams and user interface prototypes were produced for almost all of the system parts. This semester UML diagrams were used by developers when implementing functionality and were updated when functionality changed. Diagrams were also used when collecting metrics in order to more easily identify problems with the code and overall progress of the development.

CodeCrawler, 2004

Page 6 of 10

Code Crawler Final Report UPDATE DESIGN DIAGRAMS?

Version: 1.0 Date: 10/26/2013

8.

Continuously Verify Software

All documents were reviewed by group members, and continuously improved. As outlined in the Test Plan, we used testing on several levels: - Unit tests to verify lower level units of code. Most classes in the system have unit tests (black and white box) based on the JUnit framework associated with them. Unit tests were also added to verify bugs found in the software. Tests can be found under package org.CodeCrawler.test.*. JUnitEE integration was also added to allow run-time server-side execution of the JUnit tests. - Functional tests to test higher level functionality we used Selenium which is based on the idea of FIT testing. Those tests were used to verify use case scenarios for the system as a whole. - Performance tests to measure scalability and performance of the system. We used HttpUnit, which allows emulating browser behavior from within the program, to simulate multiple user accesses. We used Issuezilla as a bug and request tracking system. The roles and procedures for reporting problems and using Issuezilla that we followed were outlined in the report for HW2.

9.

Control Changes to Software

The Configuration Plan was developed early last semester and was followed throughout the development. We used CVS as a version control system of source code, Tigris.org as an archive for project documents, and Issuezilla as bug tracking system. See the Configuration Management Plan for details. Web Site configuration roles

10.

Functionality

The design of the system follows the package decomposition of the system from the Software Architecture Document and design specification from the CodeCrawler Design Document. Most of the planned functionality was successfully implemented with some features added and some postponed for future releases or scaled back. Authentication use tomcat Indexing while waiting to swap indices Scoring File date filters Modifications in CTAGS Use of term vectors for hits Highlighting of term hits

CHANGES IN THE DESIGN

11.

Metrics

Omar write a description

12.

Transition

12.1 Install Wizard Mohan, write a description

CodeCrawler, 2004

Page 7 of 10

Code Crawler Final Report 12.2 Web Site Web site plan Search engines 12.3 Press Release

Version: 1.0 Date: 10/26/2013

Martin, write a description Press Release Lucene, etc.

13.

Documentation

Each document has one or two persons responsible for maintaining it, but several people have contributed to many of the RUP documents. Additionally, everyone has reviewed all documents, and the group members responsible for the document have incorporated their comments and suggestions. Questions and input concerning the project were a result of the review of the documentation, and were discussed during the bi-weekly meetings and via team emails. The documents are based in part on the RUP document templates by IBM. Each document keeps a history of revisions and people responsible for them. 13.1 Source Code The source code archive contains external libraries, binary files, as well as CodeCrawler files. The latter include: C++ source code in /src/cpp/inc, /src/cpp/jniformatter, and /src/cpp/jnicodeanalyzer, Java source code in /src/public_html/WEB-INF/src, web files in /src/public_html (and subfolders), build and Tomcat configuration files other files 13.2 RUP Artifacts

Vision Statement Provides an overview of the proposed project why it will be useful, who will use it, what
features it will have, what competing products exist, and why CodeCrawler is better. Nikolay, Kinis, and Jebeh created this document. (NOTE: this document is a backup of the Wiki page after Inception and includes the Visioin, the original Project Plan, and Risks)

Project Plan A general outline of the process to be used.

delivered, milestones, and deadlines.

Specifies development phases, artifacts to be

Risks A list of business, project, and technical risks that endanger the project.

Each risks severity is evaluated and a strategy for avoiding or mitigating it is suggested. The existing risks were examined and reevaluated at the end of each iteration and new risks addressed.

Iteration Schedule A detailed schedule of project activities keeping track of the status, duration, resources,
input and output artifacts, etc.

Use Case Model A detailed overview of the main use cases for the system. Configuration Management Plan describes the roles and responsibilities of various team members, and
the environment and tools used to manage the configuration. Focuses on the configuration management processes including change control and auditing. CodeCrawler, 2004 Page 8 of 10

Code Crawler Final Report

Version: 1.0 Date: 10/26/2013

Deployment Plan describes the various deployment activities, who is responsible for them, and the schedule
for the same. Also, describes the resources required for the above activities.

Integration Build Plan - describes the various sub-systems in the application and the actual build process. Glossary contains the technical terms used throughout the documentation. Software Requirements Specification (SRS) describes functional requirements, non-functional
requirements, possible design constraints, interfaces, legal and licensing issues, standards, and supporting information including user interfaces prototypes for the CodeCrawler system.

Software Architecture Document (SAD) - provides a comprehensive architectural overview of the

system, using the 4+1 RUP views to depict different aspects of the system. Provides the basic knowledge on the overall design structure and also the deployment capability of the CodeCrawler software.

Test Plan outlines the overall testing strategy for the CodeCrawler project.
of testing, and test metrics to be used in testing.

Defines test responsibilities, areas

Design contains description of the design of the system.

Web Site Plan Communication Plan Developer Setup Installation Instructions Administrator Configuration Guide Metrics Report 13.3 Complete Document List Here is a list of files that were used as scratch documents and contributed to our understanding the system and creating the RUP artifacts above: File rup_wd_tmpl.zip Architecture\Software Architecture Document V1.3.3.doc Architecture\Architecture - old.zip CM\Configuration Management Plan.doc CM\Deployment Plan.doc CM\Integration Build Plan.doc Design\Design - old.zip Design\Designv2.0.doc Glossary\Glossary.doc HW2\HW2.doc HW2\hw2.vsd HW2\hw2_2.vsd HW2\hw2_4.vsd HW2\HW2-kinis,david.zip HW2\Hw2-kojuharo, mailloux.zip Description RUP templates Final Software Architecture Document Old versions and scratch documents used in the architecture Final Configuration Management Plan Final Deployment Plan Final Integration Build Plan Old versions and scratch documents used in the design Final Design Final Glossary scratch scratch scratch scratch Homework 2 files Homework 2 files Page 9 of 10

CodeCrawler, 2004

Code Crawler Final Report

Version: 1.0 Date: 10/26/2013

HW2\Hw2-mkishore, fancher2.zip Homework 2 files HW2\HW2-wisely, kawah.zip Homework 2 files HW3\HW3-awisely-kawah.zip Homework 3 files HW3\HW3_kojuharo_mailloux.zip Homework 3 files HW3\Mohan-Edward-hw3.zip Homework 3 files HW3\peerenbo_kmeyer_hwk3.zip Homework 3 files HW4\HW4 - edward, mohan.zip Homework 4 files HW4\HW4_david_kinis.doc Homework 4 files HW4\HW4_jebeh_adam .doc Homework 4 files HW4\HW4_kojuharo_mailloux.zip Homework 4 files HW5\3rd Party Tools.doc scratch HW5\Brief Introduction to CodeCrawler.doc scratch HW5\Code Crawler Glossary Text.doc scratch HW5\Code Crawler Risk Management.doc scratch HW5\Component.gif scratch HW5\Deployment-Component.vsd scratch HW5\Deployment.gif scratch HW5\Mohan (N).vsd scratch HW5\realizations.doc scratch HW5\HW5 CodeCrawler.doc Homework 5 HW5\Software Architecture.vsd scratch Meetings\Meetings.zip Meeting notes Planning\CodeCrawler Meetings and Milestones.doc Obsolete time allocation plan Planning\Project Plan.rtf Final Project Plan Planning\Risks.xls Final Risk Assessment Planning\Schedule.xls Final Iteration Plan (for all past iterations) Requirements\SoftwareRequirementsSpecificationV1_9.doc Final Software Requirements Specification Requirements\Use Cases - old.zip old use case documents Requirements\Use Cases.doc Final Use Case Model Testing\Testing - old.zip old test plan documents Testing\Test Plan v 2[1].0.doc Final Test Plan UI\code.mht Final prototype for viewing source code UI\Config UI.gif Initial prototype for configuration UI UI\logo-cc.jpg Final project logo UI\logo-cc.psd Final project logo (Photoshop format) UI\logo.psd old project logo UI\results.jpg Final prototype for results page UI\results.mht prototype for results page in MHTML format UI\search.jpg Final prototype for search page UI\search.mht prototype for search page in MHTML format UI\search_adv.jpg prototype for advanced search prototype for advanced search in MHTML UI\search_adv.mht format UI\UI-State.vsd UI state diagram Vision\SourceCrawler draft.doc Draft vision document (old project name) Final Vision statement (with inception Vision\Vision.pdf project plan and risks)

CodeCrawler, 2004

Page 10 of 10