Find us on GitHub

A Data Carpentry Workshop

Lawrence Berkeley National Laboratory

Sep 19-20, 2017

9:00 am - 4:30 pm

Instructors: Erin Becker, Asli Uyar

Helpers: TBD

General Information

Data Carpentry workshops are for any researcher who has data they want to analyze, and no prior computational experience is required. This hands-on workshop teaches basic concepts, skills and tools for working more effectively with data.

We will cover Introduction to the Shell and Cloud Computing, The Shell and Project Setup, Know Your Data and Variant Calling and Workflow Automation. Participants should bring their laptops and plan to participate actively. By the end of the workshop learners should be able to more effectively manage and analyze data and be able to apply the tools and approaches directly to their ongoing research.

Who: The course is aimed at graduate students and other researchers.

Where: 1 Cyclotron Road, Berkeley, CA 94720 Building 71B, Room 290. Get directions with OpenStreetMap or Google Maps.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating sytem (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below). They are also required to abide by Data Carpentry's Code of Conduct.

Contact: Please mail for more information.

Day 1

Morning Introduction to the Shell and Cloud Computing
Afternoon The Shell and Project Setup

Day 2

Morning Know Your Data
Afternoon Variant Calling and Workflow Automation

We will use this Etherpad for chatting, taking notes, and sharing URLs and bits of code.


Please be sure to complete these surveys before and after the workshop.

Pre-workshop Survey

Post-workshop Survey

Preliminary Schedule

Tuesday, September 19

MorningIntroduction to the Shell and Cloud Computing
09:00 Workshop overview
09:30 Using cloud computing for genomics
10:00 Introducing the Shell
10:30 Coffee/tea break
10:45 The Filesystem
11:25 Working with Files
12:00 Lunch on your own
Afternoon The Shell and Project Setup
13:00 Redirection
13:45 Writing Scripts
14:30 Coffee/tea break
14:45 Loops (if we have time)
15:45 Project Organization
16:20 Project Planning
16:45 Wrap-up

Wednesday, September 20

Morning Know Your Data
09:00 Introduction to the workshop dataset
09:15 Examining data on the NCBI SRA database
09:45 QC of NGS data
10:45 Coffee/tea break
11:00 QC of NGS data (cont)
12:00 Lunch on your own
Afternoon Variant Calling and Workflow Automation
13:00 Variant calling workflow (QC, alignment, variant calling)
14:00 Automating the workflow
14:30 Coffee/tea break
14:45 Automating the workflow (cont)
16:45 Wrap-up


To participate in a Data Carpentry workshop, you will need working copies of the described software. Please make sure to install everything (or at least to download the installers) before the start of your workshop. Participants should bring and use their own laptops to insure the proper setup of tools for an efficient workflow once you leave the workshop.

Mac users

Windows users

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.