Rework the documentation hosting & generation and Restructure getting started pages and developer guides

logo-lf,agl

Table of Contents

  1. Introduction
  2. Organization Information
  3. Project Interested
  4. Why this project?
  5. Deliverables
  6. Project Description
  7. Timeline
  8. Commitment & Availability
  9. Support required
  10. Post GSoD Period
  11. Open Source Contributions & Experiences

Other Versions :

Contact Information

  1. Full Name : Shankho Boron Ghosh (resume)
  2. Email : shankhoghosh123@gmail.com
  3. Country : India
  4. Github Username : growupboron
  5. LinkedIn : https://www.linkedin.com/in/shankho-ghosh/
  6. IRC Nick : boron
  7. Preferred Communication : Email, Video Conference
  8. Website : https://growupboron.github.io/

1. Introduction:

  • I am Shankho Boron Ghosh, a sophomore at Thapar Institute of Engineering & Technology, Patiala (India). I am doing my majors in Electronics & Communication Engineering with minors in Computer Science.
  • I have project-based experience in Python, C, C++, Robot Operating System (ROS), tensorflow, OpenCV and IoT using Arduino, esp8266 and Raspberry Pi. Being a quick learner and a natural improviser, and having applied this skill of problem-solving to win multiple national level hackathons including the prestigious Smart India Hackathon.
  • I recently interned as a Robotics Research Intern ​ at the Indian Institute of Information Technology, Allahabad. I am also the On-Board Computer engineer​, for the Student Satellite Team of our university, ThapSat and the ​Data Acquisition Lead for the Formula Student Team, FSAE Team Fateh at my university.
  • I have a good understanding and experience of software and hardware engineering related documentation methodologies, tools and usage.

2. Organization Information: The Linux Foundation

The Linux Foundation is the nonprofit consortium dedicated to fostering the growth of Linux. Automotive Grade Linux is an open source project hosted by The Linux Foundation that is building an open operating system and framework for automotive applications.

3. Project Interested: (idea page)

  • Rework the documentation hosting and generation
    • The doc pages are currently hosted within the individual sources as markdown using the engine of the cordova project. Rework to documentation into a single repository setup (preferably markdown) that can be used e.g. by readthedocs or similar.
  • Restructure getting started pages and developer guides
    • The Getting Started pages on doc site are the entry point for new users. The goal is to make these clear and bullet-proof for the reader. This involves checking the consistency and flow of the documentation and making sure things are in the right sequence.

4. Why this project?

  • Having first encountered Automotive Grade Linux when we (Team Fateh) had to decide on a robust platform that could handle low latency data communication and be reliable enough for automotive data acquisition (on the Raspberry Pi prototyping board), unlike the out of the box Raspbian Operating System, as the part of my work as Data Acquisition Engineer at the Formula Student Team, FSAE Team Fateh at my university. Correspondingly, the principle on which Automotive Grade Linux works fascinated me.
  • Having previously used AGL (on Raspberry Pi) for designing a Data Acquisition System, it’s very clear to me, the minute specificity or the refinement needed from a user’s perspective and relevant desired knowledge for the project to improve the documentation regarding the same.
  • Furthermore, I have relevant experience in documentation in the following ways :
  • I have also been participating and actively interacting and engaging in the AGL community, regularly attending the Weekly Developer Calls, and further understanding the organization’s development and production pipeline and developer’s tool and practices.
  • Local development system specifications :
  • I truly believe Open Source is the future and the best technique of learning is by doing. The perks, opportunities and prestige that are associated with Google Season of Docs (GSoD) are just added benefits to this.
  • This project would give the necessary experience and rigor to contribute meaningfully and efficiently as Data Acquisition Engineer to FSAE Team Fateh.

5. Deliverables:

  • The primary aim of the project is to simplify the existing AGL documentation workflow pipeline by reworking the documentation hosting and generation :
    • Develop a single repository setup from the current four repository setup to improve maintainability of the documentation site.
    • Simplify the indexing of the existing markdowns by using directory tree structure instead of maintaining cumbersome yml indexes.
    • Migrate the whole setup from the current Jekyll template to MkDocs static site generator which can be hosted at readthedocs which simplifies :
    • This setup would vastly improve and simplify fetching the markdowns of different releases of AGL from multiple repositories hosted mostly on gerrit into different branches on a single documentation repository setup.
  • Improve the Getting Started page and Developer guides the existing documentation along with writing new documentation into the following clear and bullet-proof structure :
    • Tutorials: a lesson that allows the newcomer to get started.
      • Example : teaching a small child how to cook
    • How-to Guides: a series of steps that show how to solve a specific problem.
      • Example : a recipe in a cookery book
    • Reference: a technical description describing the appropriate machinery and workflow processes.
      • Example : a reference encyclopedia article
    • Explanation: discursive explanation explaining code / workflow structure in detail
      • Example : an article on culinary social history
  • Rewrite the documentation of existing “How to handle documentation” to the proposed setup and with the aim of making it beginner and developer friendly.
  • Ultimately, making the documentation community driven so users can report any issues of the documentation and also be able to make changes that will be reviewed and merged by the AGL team.

6. Project Description:

  • Abstract :
    • A documentation is designed to assist end users and developers to use a product or service. Good documentation is very important because it provides an avenue for users to learn how to use a software, its features, tips, tricks and also resolve common problems encountered when using the software. It also reduces support cost and is part of the corporate and open source identity of the product : a good documentation is a sign of healthiness of the product and the developer team.
    • Without a good documentation, a user may not know how to do the above things effectively and efficiently. Documentations can play a pivotal role in ensuring a product’s success because great communication is and will always be at the heart of any business or product and a great documentation just takes that communication and puts it in a manageable framework that everyone can access for success.
    • Every documentation site needs a good building and hosting workflow pipeline, in an organization like AGL, with multiple versions and a lot of elaborative documentation, the documentation files (markdowns) are spread across multiple repositories, making the task of maintaining and updating them incredibly complex and time intensive.
  • Current State :
    • AGL doc website is based on a collection of markdown files fetched from various repositories.
    • The doc pages are currently hosted within the individual sources as markdown using the engine of the cordova project.
    • This leads to a four repository setup for the documentation build and hosting process :
    • A tool (script) available in docs-tools takes care of collecting and templating all markdown files according to the fetched_files.yml located in docs-webtemplate.
    • The current workflow of agl documentation website generation : current_workflow
    • The section_version.yml contains the links to all the book yaml files, it proceeds to fetch all book yaml files from remote repositories to the docs-webtemplate. The book yaml files contain all the url to your markdown files from the remote repository.
    • As soon as all the markdown files are fetched, the tools process to generate the AGL doc website in the docs-gh-pages which is correspondingly deployed.
    • The current process of maintaining the pipeline is not user and developer friendly, especially to new contributors. This workflow pipeline (of building and hosting) can be simplified and streamlined way more for developers to focus on the documentation part rather than maintain documentation generation and deployment workflow.
  • Analysis :
    • Jan-Simon Möller and I had multiple conversations over the scope of development of restructuring the developer guides and reworking the documentation generation and hosting. We discussed the requirements of AGL, and the hurdles of the current complicated workflow pipeline.
    • After much research and deliberation, we mutually agreed that weighing multiple static site generators would help us compare the quirks of each site elaborately and after discussion, I didn’t just read about these tools, I took up the initiative to create and deploying two mockups :
    • The creation of the mockups helped us realize that the workflow of MkDocs + readthedocs is best suited for our needs as :
      • Mkdocs’s indexing is much easier than Docusaurus as it automatically indexes docs based on their directory’s tree structure, and this was one of the major reasons for AGL wanting to migrate off from the current Jekyll build due to the fact that maintain the indexing yml file is cumbersome and time consuming.
      • Also, Mkdoc’s implementation on readthedocs was more favourable than github pages as :

        • Readthedocs takes care of versioning in a such simplified manner by creating different branches of each version in docs folder wherein in github pages, this isn’t possible,

        • Read the Docs simplifies software documentation by automating building, and hosting of your docs for you.

        • Therefore, the proposed workflow is as : proposed_workflow

        • Also, we can later think about multiple language support or localization of documents, though this will require Markdowns to be converted into ReStructuredText for being fully compatible with Sphinx site generator.

        • Searching is more easily enabled because Read the Docs uses Server Side Search to power search.

        • The implementation would be quite similar to NodeMCU’s official documentation.

    • This proposed workflow will help AGL documentation to be migrated into a much simpler and systematic, single repository setup against the current overcomplicated four repository setup that will ultimately make the documentation community driven so users can report any issues of the documentation and also be able to make changes that will be reviewed and merged by the AGL team.
    • Improve the Getting Started and Developer guides by restructuring the existing documentation along with writing new documentation into the following clear and bullet-proof structure :
      • Tutorials: a lesson that allows the newcomer to get started.
        • They are wholly learning-oriented, and specifically, they are oriented towards learning how rather than learning that.
        • Analogy to cooking : teaching a small child how to cook.
      • How-to Guides: a series of steps that show how to solve a specific problem.
        • In a how-to guide, you can assume some knowledge and understanding. You can assume that the user already knows how to do basic things and use basic tools.
        • Analogy to cooking : a recipe in a cookery book.
      • Reference: a technical description describing the appropriate machinery and workflow processes.
        • Reference material is information-oriented and should be austere and to the point. Reference guides have one job only: to describe. They are code-determined, because ultimately that’s what they describe: key classes, functions, APIs, and so they should list things like functions, fields, attributes and methods, and set out how to use them.
        • Analogy to cooking : a reference encyclopaedia article.
      • Explanation: discursive explanation explaining code / workflow structure in detail.
        • Explanation, or discussions, clarify and illuminate a particular topic. They broaden the documentation’s coverage of a topic.They are understanding-oriented.
        • Analogy to cooking : an article on culinary social history
      • Restructuring_visualization is the visual description of the types of documentation.
    • The tutorials and how to guides are targeted toward new user, and to make new users comfortable, I plan to :
      • Use the Flesch-Kincaid score of the document at least above 50. According to this method, the higher the score, the easier the piece is to understand. You can check its readability score online.
      • Use bulleted or numbered lists so that readers do not get bored or distracted by long paragraphs.
      • Use shorter paragraphs and sentences because it requires comparatively more mental work to read and understand longer sentences.

7. Timeline:

  • Pre GSoD - 9th July - 16th August :
  • Community Bonding Period - 17th August - 13th September :
    • Take suggestions from mentors about improvement to the workflow of reworking documentation hosting & generation.
    • Plan about which specific parts of documentation to improve after reworking documentation hosting & generation.
  • Week-1 : 14th September - 20th September :
    • Setup the initial repository and link that to readthedocs.
    • Go over the whole documentation website, and make note of erroneous portions.
    • Write scripts to pull markdowns from a lot of different repositories into suitable branches and directories to automate the process of indexing.
  • Week-2 : 21th September - 27th September :
    • Test out the initial build, and list down discrepancies in the linkings of images and structure because the migration to new static site generation.
    • Write scripts to check out dead links and do necessary changes to raw markdowns for automatic generation of correct documentation.
  • Week-3 : 28th September - 4th October :
    • Host the site on readthedocs and properly setup the platform to support multiple versions smoothly.
  • Week-4 : 5th October - 11th October :
    • Take reviews from the AGL community about the new documentation site, and change the readthedocs template (if necessary).
    • Work on improving the speed of search in the documentation site.
  • Week-5 : 12th October - 18th October :
    • Work on documenting the new documentation generation and hosting process.
    • Start setting up Raspberry Pi to improve the developer guides for the same.
  • Week-6 : 19th October - 25th October :
    • Debug the Raspberry Pi build and take reviews from the AGL community about important Raspberry Pi bugs to include in the documentation.
    • Start restructuring of the Raspberry Pi documentation into different parts : tutorials, how-to guide, reference and explanations.
  • Week-7 : 26th October - 1st November :
    • Do the same for quemux86-64.
    • Finish restructuring of the Raspberry Pi documentation and take review from the community over the possibilities of improvement.
  • Week-8 : 2nd November - 8th November :
    • Improve and write content for the Getting Started page portion of documentation.
  • Week-9 : 9th November - 15th November :
    • Improve and write content for the Developer Guides portion of documentation.
  • Week-10 : 16th November - 22nd November :
    • Improve erroneous sections in current documentation.
    • Check out the Flesch-Kincaid score of readability and improve on the parts of the documentation with scores lower than 50.
  • Week-11 : 23rd November - 29th November :
    • Enhance Getting started pages and Developer guides and after reviews from the community.
  • Week-12 : 30th November - 5th December :
    • Write down a resource list that future documenters and developers can follow to write and improve documentation.
    • Write down and submit the final project report.
  • Week-13 : 5th December - 10th December :
    • Submit the personal evaluation on success of the project and experience working with AGL mentors and community.

8. Commitment & Availability:

  • I have always been keen to learn more, especially about Open Source Software Development. An opportunity like Season of Docs with The Linux Foundation seems like the perfect path for it. The mentors have been very kind, patient and helpful to all my queries. I would be honored to continue working with them.
  • I am sure that I would be able to devote 40+ hours per week to this cause. My work timings are very flexible, but I usually start working after 14:00 all the way till 23:00 in UTC+5:30.
  • I expect to give my full participation during the GSoD period and have no prior engagements during this period, except that I’m a rising junior, so there might be time when I would be unavailable due to academic commitments.
  • I would suggest we start working a little earlier than the original GSoD timeline in order to avoid any unforeseen notices by my university regarding academic examinations.

9. Support Required:

10. Post-GSoD Period:

  • I will maintain and add to the Automotive Grade Linux’s documentation repositories for the foreseeable future and work on continually improving it.
  • I will keep contributing and help other new contributors to explore and learn projects (especially the documentation) in this organisation.

11. Open Source Contributions & Experiences:

  • Contributions to Open Source :
  • As a professional tinkerer, I have actively been part of developing several projects as mentioned on my resume and GitHub.
  • Also being an avid hackathon hunter, I have participated in over a dozen hackathons collecting many laurels, helping me to become a quick learner and a natural improviser.
  • I can truly say that my experiences taught me the value of patience, perseverance, and to constantly strive for honor and excellence.
  • I believe in learning by doing and have been exposed to a lot of things and I will always treasure the insights that I’ve earned both and in and outside the portals of the university.