While California may be home to some of the most aggressively forward-thinking tech companies in the world, that enthusiasm for innovation hasn’t carried over to the public sector. State and local governments have been frustratingly slow to make public data available online. There hasn’t been anything close to a statewide standard, leaving individual agencies to voluntarily develop open data policies, often in an inconsistent and piecemeal fashion, or not at all.  

That would change if the California legislature passes two bills, S.B. 573 and S.B. 272, which would put state and local government bodies respectively on the path to open data.

The general philosophy behind open data is that datasets maintained by the government agencies are public records and should be, by default, machine-readable and downloadable from public websites. Open data allows for greater analysis and oversight by citizen watchdogs, researchers, and journalists, and creates many opportunities for civic-minded coders to create new tools for interacting with the government. 

While these bills aren’t quite as robust as transparency zealots like us would prefer, they do put California on the right track to greater transparency, accountability, civic engagement, and innovation by expanding public access to government data.

S.B. 573 – State-level Open Data

Sen. Richard Pan’s S.B. 573 would establish the position of Chief Data Officer for the state of California. This executive officer would be tasked with creating an open-data roadmap, open-data guidelines, an open-data working group, and a statewide open-data portal. 

Currently, the California Attorney General’s office has provided guidelines used statewide for compliance with the California Public Records Act.  However, the California Department of Justice isn’t particularly well suited for advocating open data, as evidenced by our recent battle over the office’s short-lived policy of only providing wiretap data in a “locked PDF” format.  Creating a new officer whose primary job is to research and promote open data will go a long way to address this issue.

Of course, the bill could be much better.  For example, the timeline described in the bill would require the state government to publish 150 datasets by June 1, 2017. That’s a pretty mediocre commitment, considering that data.ca.gov already links to more than 120 datasets. The state could move much faster.

We’ve also urged the legislature to strengthen the bill by including non-profit transparency groups in the open data working group. We would also like to see the Chief Data Officer tasked with ensuring that data is accurate and complete, in addition to being available to the public.

We’re also glad that the bill specifically mentions privacy, because open data can make it easier to identify individuals even after attempts to strip identifying information like name, gender, address, birth data, phone number, and Social Security number.  In our letter supporting the bill, we asked that the Chief Data Officer and the open-data working group evaluate potential “reidentification” effects, which have been of particular concern for public health data. 

S.B. 573 passed out of the Senate with a strong majority and will be heard on July 7 in the Assembly’s Privacy and Consumer Protection Committee

S.B. 272 – Local Government Data Inventories

Sen. Robert Hertzberg’s S.B. 272 would require local government agencies to publish a list of all the information systems ("enterprise systems") they maintain. These catalogs of data would include basic details for each database, including a description of the purpose for the system, how the data is collected and updated, and the vendor providing the software or hosting for system. This last point is important for ensuring accountability in this age of outsourcing.

However, this bill does not require local agencies to actually publish the dataset themselves online, although the legislation does serve as a stepping stone to potential future expansion similar to S.B. 573 (in fact, local agencies would have access to the same tools developed by the Chief Data Officer). In the meantime, these inventories would serve as a menu of sorts from which the public can request data through a California Public Records Act request. 

We’re not thrilled that the legislature has exempted school districts and other local education agencies from the bill’s requirements. We see no reason that the public should not be able to learn basic information about how schools and their vendors collect and store information about students and teachers, especially as private “cloud” companies become more involved in schools.

S.B. 272 passed unopposed in the Senate and is set for a hearing on July 15 in the Assembly’s Committee on Local Government. 

Update: On July 6, S.B. 272 was amended, changing "information systems" to "enterprise systems," and "school districts" to "local education agencies." We have adjusted the post accordingly.