Sunlight Labs creates catalog of government databases

From Federal Computer Week, comes this note:

The Sunlight Labs open government group recently debuted its free online open-source collection of federal, state and local government databases in the United States.

Despite its ambitious name, the group’s National Data Catalog currently only has links to data from the federal Data.gov Web site, as well as data from the District of Columbia and the state of Utah.

Sunlight’s goal is to continue to post more databases and links to federal, state and local datasets. The national data catalog platform consists of a Web application and an Application Programming Interface. Users can register to obtain their own dashboard allowing them to search and examine the data.

For more, see the FCW post.

Federal Agencies Announce Open Government Plans

Federal departments and agencies released their Open Government Plans today. The Plains aim to make operations and data more transparent and increase opportunities for citizen oversight of the government.

Examples of each agency's "flagship initiative" include:

• Department of Health and Human Services' Community Health Data Initiative: The department will provide a free, large-scale Community Health Data Set including downloadable data on community health care costs, quality, access, and public health.
• Department of Energy's Open Energy Information Initiative: The department will make more than 60 clean energy resources and data sets, including maps of worldwide solar and wind potential, information on climate zones, and best practices available.
• Department of Housing and Urban Development's Homelessness Prevention Resources Initiative: The department will develop tools to predict communities that are at risk so that resources can be allotted to prevent homelessness.

For more information, click here.

Is Obama's Open Government Initiative Technologically Feasible?

Obama's open government directive will include a schedule for data distribution in formats that are machine-readable, says the Federal Chief Technology Officer Aneesh Chopra.

But some e-government interest groups say that's unrealistic because most agency data exists in PDF format, which is not machine-readable and from which data cannot be easily extracted.

That makes the transfer of legacy data difficult, but one e-government advocate says newly created data could be stored in a new PDF/A version that is more suitable for long-term preservation.

For more information, click here.

FOIA Request Shows Gaps in ICE Detention Records

Based on records obtained by the Associated Press through FOIA requests, the Washington Post reports that think tank Migration Policy Institute has identified several gaps in the record-keeping of the U.S. Immigration and Customs Enforcement.

These gaps include:

* whether an immigrant must be detained;
* whether a detainee is dangerous;
* whether a detainee might have a claim to U.S. citizenship; and
* whether a immigrant has a special medical condition or mental health issues.

The think tank also says ICE's database should include answers to procedural questions that would ensure the federal government is not holding the immigrants indefinitely.

U.S. Government CIO speaks about plans for Data.gov

Wired published a Q&A with Vivek Kundra, the U.S. government's first-ever chief information officer, in this month's issue. Kundra spoke about his plans for Data.gov and navigating privacy issues. Wired also has its own Wiki where people can report government data that is not accessible and can suggest how accessibility to government data could be improved.

The Obama administration's most radical idea may also be its geekiest: Make nearly every hidden government spreadsheet and buried statistic available online, all in one place. For anyone to see. Are you searching for a Food and Drug Administration report that used to be obtainable only through the Freedom of Information Act? Just a mouseclick away. Need National Institutes of Health studies and school testing scores? Click. Census data, nonclassified Defense Department specs, obscure Securities and Exchange Commission files, prison statistics? Click click. Click. Click.

The man in charge is the US government's first-ever chief information officer, Vivek Kundra. Previously CTO of the District of Columbia, Kundra, 34, knows that the move from airtight opacity to radical transparency won't be a cakewalk. Until now, the US government's default position has been: If you can't keep data secret, at least hide it on one of 24,000 federal Web sites, preferably in an incompatible or obsolete format.

More here.

NFOIC Summit: Infrastructure coverage tips

James Shiffer, reporter and editor at the Minneapolis Star Tribune; Tisha Thompson, investigative reporter at WTTG Fox 5 in Washington, D.C.; and Jaimi Dowdell, IRE training director and former computer-assisted reporting editor at St. Louis Post-Dispatch, discussed how to use FOI to inform the public about infrastructure issues. Patrice McDermott, director of OpenTheGovernment.org, moderated. Here's part two of the notes from their session at the 2009 NFOIC Summit:

Tips for infrastructure coverage:

• Look at the annual report card put out by the American Society for Civil Engineers each year.
• Keep the National Bridge Inventory and National Inventory of Dams database handy.
  
• Use electronic databases to lead you to paper documents.
• When looking at infrastructure records, contact state inspectors and civil engineers. They can help you analyze data so you don't make mistakes. If possible, first contact other reporters who have done similar stories in other cities.
• The smaller the town or jurisdiction, the better chance you'll have of getting data. Bigger cities become savvier.
  
• Ask for five years of inspection reports to notice problems that haven't been fixed.
• Start with bridges and dams stories. River communities should look into levees.
• Keep a list of those you request records from. Divide them into a naughty and nice list. Send Christmas cards to both. Be creative.
• Go to your local transportation department and goggle what it has posted.
• Look at the infrastructure in urban sprawl areas.
• For a breaking news story, immediately go to the web and take screen shots of everything you can find about the bridge, dam, etc. Chances are that someone will take that info down.
• When googling something, click on the Google cached link, which Tisha Thompson refers to as the "sexiest thing in the world." It shows you what a Web page looked like on a certain day.
• You can also use Wayback Machine to find older versions of Web pages. Thompson used this to access the Web site of Eliot Spitzer's call girl after her site was removed. She was able to obtain information about her and her picture. Warning: This site doesn't delve deeply into federal and state government Web sites.
• Use your stories to highlight the ridiculous excuses given for why information isn't made public.
• Find the human element. Remind agencies that people have a right to this information. This is the people's money and safety at stake.
• Find an inspector who cares and convince him/her to provide you with data or the records.
• Get data any way you can, even if it's in a PDF file. Then, find someone who can figure out how to turn it into something usable.
• Trick from Jaimi Dowdell: If a government agency says it can't export data because of the system it's using, just say "Well, that's terrible. But I have to give my editor a report back on why I couldn't get this data. What's the name of your software?" Then call that software company and tell them that their client says it can't export the data. Chances are, the software company will say its software can do back flips. It usually contacts the clients and essentially helps you get the data.
• Conduct advanced searches in Google and search by file type.
• Fish for data and know what's out there. Agencies can't claim records don't exist that way.
• Consider looking into highway construction as well as asphalt and concrete contracts.
• Look into data on electricity grids, fire hydrants, street light outages, pot holes, population growth, water main breaks, road construction, etc. Who's getting stimulus funding?
• Request videos and photos in your FOI requests.
• Find more tips here.

Obama follows Bush in keeping details of FBI database secret

President Barack Obama refused to reverse a Bush policy that protected information about the FBI's Investigative Data Warehouse. "As a result, there is no public list of all the databases the FBI sucks into this computer warehouse; no information on how individuals can correct errors about them in this FBI database; and no public access to assessments the bureau did of the warehouse's impact on Americans' privacy," the AP reported. The warehouse is five years old and contains more than 50 databases.

Despite a pledge to open government, the Obama administration has endorsed a Bush-era decision to keep secret key details of an FBI computer database that allows agents and analysts to search a billion documents with a wealth of personal information about Americans and foreigners.

President Barack Obama's Justice Department quietly told a federal court in Washington last week that it would not second-guess the previous administration's decisions to withhold some information about the bureau's Investigative Data Warehouse.

The Electronic Frontier Foundation, a digital rights advocacy group, had sued under the Freedom of Information Act to get records showing how the FBI protects the privacy of Americans whose personal information winds up in the vast database.

More here.

Big victory could mean public right to any government-created database

A California appeals court ruled that Santa Clara County must make its geographic information system, which indicates real estate boundaries and other property data, available to the public. The county had denied the California First Amendment Coalition's request for a copy of the real estate parcel "basemap." The argument was that U.S. Homeland Security concerns and copyright protection preempted releasing the information in accordance with the California Public Records Act.

In a stunning victory for freedom of information advocates, a California appeals court has sustained a public right of access to a government database that makes possible highly accurate digital mapping.

Holding that Santa Clara County must make public its geographic information system (GIS) parcel “basemap,” which shows real estate parcel boundaries, appraisals, and other property data, a unanimous Court of Appeal rejected arguments that U.S. Homeland Security regulations and Federal Copyright protection trump the public’s right of access under the California Public Records Act.

More here.

More coverage: Homeland Security Act Does Not Shield County Land Data, Metropolitan News-Enterprise