Written by Ines Aisengart Menezes and Yvonne Ng
At WITNESS, one common question we hear from partners and community-based human rights video practitioners that we support on archiving is “What database tool should I use to manage my videos?”. It’s a tricky question, because there isn’t one single correct answer. Usually, our reply is, “It depends!”
Many factors can influence your choice of cataloging or database tool – cost, technical requirements, community engagement and collaboration features, usability and accessibility, support and training, scalability and sustainability, and security and privacy for sensitive data.
We composed this blog post to compare four cataloging / database tools that we have used previously in our projects, according to various key factors. This short list is far from all the options available, and they may not be the most suited ones for you – we encourage you to examine other tools, perhaps using the categories we’ve used here!
Remember, no matter what tool you choose, the key to building an effective caalogue is to have a good underlying data model, i.e. the way you structure your database, the elements/fields you use, and how they relate to one another. This applies whether you are using a complicated database app or a simple spreadsheet. It is also important to document your data model in a data dictionary, to ensure that cataloging is performed consistently and correctly, and to have a strategy for incorporating cataloging into your workflow.
For more tips on how to structure data, check out our Quick Guide to Video Archiving.
Uwazi
Uwazi is an open-source web-based database app created and maintained by the NGO HURIDOCS (Human Rights Information and Documentation Systems). We are using it in our Rohingya Genocide Archive initiative with Rohingya Vision. Uwazi is used by NGOs, journalists, academics, lawyers, and others for documenting human rights violations, managing documents, and case/complaint management.
Cost and Requirements
Uwazi is open-source, so it can be downloaded and installed for free. Self-hosting Uwazi, however, requires a server and a server administrator (e.g., for configuration, installing dependencies, maintenance, running updates, backup, etc.).
An easier option is to have a Uwazi instance hosted and maintained by HURIDOCS. HURIDOCS offers different subscription packages depending on the volume of data. Catalogues under 100 MB are free, while catalogues up to 500 MB are approximately US$100/year. Most text-based catalogue databases will not exceed this size (note: audiovisual content needs to be stored separately and linked).
Community Engagement and Collaboration
Uwazi allows multiple users to collaborate on creating or using a catalogue. Users can be assigned different permissions (Admin, Editor, or Collaborator levels) and can be organised into user groups for easier management. There is also a submission form feature for gathering information from external community members.
Usability and Accessibility
Uwazi has a fairly user-friendly interface, but configuring your Uwazi database requires some data modelling skills and review of the Uwazi Administrator’s guide. It is also possible to contract with HURIDOCS, as we did, to have them develop the system based on your needs.
The Uwazi interface is available in English, Arabic, Burmese, French, Korean, Russian, Spanish, Thai and Turkish, although it is also possible for admins to manually translate and re-label interface objects (i.e. buttons, menus, etc) into other languages.
Uwazi instances hosted by HURIDOCS or in the cloud require the internet to access and use.
Scalability and Sustainability
HURIDOCS actively maintains Uwazi’s technical infrastructure and performs ongoing development. It sustains this work through its hosting fees. While Uwazi is open-source, there is a risk that, as a somewhat niche community tool, it may not be sustainable if HURIDOCS were to stop maintaining it.
Security, Privacy, and Sensitive Data
Of the cataloging tools we review in this list, Uwazi is the only one created and maintained by a human rights organisation, intended for use by the human rights community. As such, Uwazi has been built with information security in mind. HURIDOCS uses data centres in the Netherlands run by Greenhost, with dedicated security infrastructure including data encryption, monitoring, and firewalls. HURIDOCS publishes weekly feature updates and security patches and pushes them to hosted instances. There are also options for hosted setups that use VPN and TOR. Hosted instances are backed up on a schedule in an encrypted storage facility.
On the user side, Uwazi includes options for public and private collections that require user login and optional multi-factor authentication. Admins can monitor for suspicious activity via activity logs.
Why We Chose It for the Rohingya Genocide Archive
For this project, we needed a tool that could support a remote and decentralized team of catalogers. The security of our catalogers and of the data was a significant concern, so we wanted a tool that incorporated an awareness of the risks faced by human rights documenters.
We also wanted access controls that could accommodate different levels of catalogers, enable selective access to external users, and allow for eventual controlled public access. We had a budget for this project, and wanted assistance with setting up and configuring a hosted instance, which we were able to obtain through a contract with HURIDOCS. We have also known HURIDOCS for many years as a peer organization, so there was already some pre-existing trust. One shortcoming for us in using Uwazi is that it was not designed for cataloging or managing video collections, so its video-based features are minimal. For example, videos need to be embedded into records from other hosts like YouTube or Facebook.
FileMaker
FileMaker is a commercial relational database app by Claris, an Apple subsidiary. We used it to create the People’s Database for Community-Based Police Accountability with Berkeley Copwatch. It combines a backend database engine with a graphical user interface (GUI) so that users can create databases through pointing-and-clicking and filling in dialog boxes, rather than coding. FileMaker is used mostly by small organisations and small businesses in the US. Since users build their own databases from scratch, it can be used for any purpose, including cataloging human rights videos.
Cost and Requirements
“FileMaker Pro” is the standalone desktop app for your computer, and is the most basic way to use FileMaker. There are other versions of FileMaker including “FileMaker Server” and “FileMaker Cloud,” where you can host your own database or have it hosted by Claris.
As of April 2023, there are different licence / subscription fees for each version of FileMaker:
- FileMaker Pro: A single licence for one computer at a time is a one-time cost of US$594.
- FileMaker Server: Licences for 5-99 users start at US$2,970 for a perpetual licence, or US$990 per year for an annual licence. Includes FileMaker Pro.
- FileMaker Cloud: Licences for 5-10 users start at US$1,260 per year. Includes FileMaker Pro.
See detailed operating system and hardware requirements here.
Community Engagement and Collaboration
The standalone FileMaker Pro licence will only allow you to have one local instance of FileMaker running at a time (although you can create multiple user accounts in that instance), while FileMaker Server and FileMaker Cloud support having multiple simultaneous remote users.
Data can be easily exported from FileMaker into common formats like CSV and XML for sharing.
Usability and Accessibility
While FileMaker has a relatively friendly user interface, it has a lot of features and it is necessary to read the documentation or follow some tutorials to learn how to use it. FileMaker is a “blank slate,” so users will need to spend some time planning and designing their database before building it in FileMaker. Depending on how complex your database is, and how you want to work with your data, it may be helpful to have data modelling, scripting, and graphic design skills. FileMaker does offer some standard basic templates, and it’s possible for anyone to create and share their template, such as the People’s Database for Community-Based Police Accountability template that we created with Berkeley Copwatch.
FileMaker databases need to be built using FileMaker Pro on a computer, but can then be accessed and used on an iPhone/iPad or in a web browser if you have FileMaker Server or FileMaker Cloud. FileMaker Cloud requires internet access, but FileMaker Pro and FileMaker Server do not.
FileMaker supports English, French, Italian, German, Swedish, Japanese, Dutch, Spanish, Simplified Chinese, Korean, and Brazilian Portuguese.
Support and Training
Because it is a widely used commercial tool, FileMaker has extensive formal documentation available online, plus many more informal sources of information on various community user fora on the web. In our experience, Googling our FileMaker questions usually works too.
In terms of development and training, there are many independent third-party FileMaker consultants available for hire. Claris provides a list; we used the organisation join::table to connect with developers for our People’s Database project.
Scalability and Sustainability
FileMaker is the most expensive option on our list, in terms of upfront software licensing costs, making it an unrealistic option for many. Moreover, licensing is priced on a per user basis, so the cost increases if you add more users. For those who can afford the cost, FileMaker is likely a sustainable option – it has been around for a long time, is continuously maintained and supported, and is widely adopted.
In terms of scalability, FileMaker can technically support large databases and a large number of concurrent users, but this also depends on which version of FileMaker you are using and your pricing plan.
Security, Privacy, and Sensitive Data
Access within FileMaker is controlled using user accounts, which can be set on a very granular basis with highly customizable privilege sets.
Optionally, you can encrypt your database file locally on your computer (which can then be uploaded to the cloud if you are using FileMaker Cloud). Be aware that there is no way to recover your decryption password if you lose it (i.e. you will not be able to open or access your database anymore).
Depending on your threat model, using the standalone version of FileMaker Pro could be a safe option because it is completely offline.
Why We Chose It for the People’s Database for Community-Based Police Accountability
FileMaker template of the People’s Database for Community-Based Police Accountability.
Berkeley Copwatch wanted a database that could track different types of information, including incidents, videos, and police officers, so a relational database was ideal. They also needed a database with a very user-friendly and customizable interface that their volunteers and researchers could be easily trained to use. Finally, because of security concerns, they wanted a database that was only accessible from their office and that they could host themselves, but that did not require a lot of IT support. Berkeley Copwatch had also used FileMaker previously, and had older incident records that could be imported into a new database.
Tainacan
Tainacan is a flexible open-source platform for digital collections and repositories that can be used, copied, modified, and redistributed without any restrictions as a plugin in WordPress. Our collaborators, the Brazilian Association of Audiovisual Preservation (ABPA) chose Tainacan as the tool for their collaborative repository of audiovisual preservation resources. Tainacan has been developed since 2014 by Brazilian universities, and although initially aimed for cultural and museological collections, like the Brazilian museum heritage platform Brasiliana, many human rights projects are adopting Tainacan, like Centro de Memória Queixada and Memorial da Resistência. Special metadata features include geolocation (to view items on a map), and taxonomy (a controlled hierarchized list of terms), and it’s possible to use DublinCore as a metadata model for data exchange. There are multiple ways to customise filters and visualise the collections. It’s possible to connect items in the same or different collections.
Cost and Requirements
Although Tainacan is free and has no installation or upgrade costs, it runs on WordPress and depends on WordPress hosting, which can be provided either through a web hosting service (costs according to local providers). Alternatively, if you have some IT skills, you can install a web server and WordPress on your own computer and run Tainacan offline.
Community Engagement and Collaboration
The configuration for users in Tainacan is similar to the WordPress roles and permissions, with possible customizations. The log system allows moderators to track changes.
Relevant features for community engagement include the ability to accept external submissions that can be approved before publishing, and the ability of external users to contribute new terms to a controlled list of terms. It’s also possible for users to comment on each item, which can be enabled for anonymous or identified users.
Tainacan’s source code is available on GitHub, where users can collaborate on its development. After setting up the metadata scheme, uploading items via CSV files is possible, allowing importing and exporting databases.
Usability and Accessibility
The Tainacan interface is highly accessible and easy to use, with a straightforward dashboard and functionalities. Installing plugins within the WordPress dashboard and setting up the collection is quick and intuitive, and does not require expertise in any programming language.
The Tainacan plugin in WordPress is available in English, Portuguese, and Spanish; and there are versions created by users in Albanian, Catalan, French, Greek, Slovak, Spanish (from Colombia, Mexico, Spain), Swedish, and Ukrainian, with different levels of development. With a Google Translate plugin it is possible to present content in several different languages.
Tainacan has over 1.000 active installations worldwide, and over 110 installations are documented in a Tainacan repository (currently available in Portuguese) – mainly in Brazil, and also Chile, Greece, Mexico and the US.
To display Tainacan public pages with their full set of features (faceted search and media gallery, for example), users can use one of the WordPress themes supported by the Tainacan community, or implement templates via code or blocks that are available for the WordPress editor.
Support and Training
Tainacan offers comprehensive Wiki documentation in English, Brazilian Portuguese, and Mexican Spanish – although a few pages still need a translation. The active online forum provides extensive guidance to assist with configuration, where users can ask for help or report bugs with multilingual support. There are several webinars for setting up Tainacan repositories in Portuguese, Spanish and English.
Scalability and Sustainability
The steady investment from developing universities makes Tainacan a stable and sustainable tool for the community. Despite being a tool built and consolidated since 2014, as of May 2024, the latest version published is 0.21.2. Some final details and revisions are being made so that the 1.0 can be launched.
Security, Privacy, and Sensitive Data
Data security depends in large part on the measures taken by your website host or on your local server installation. However, there are also steps that Tainacan users and administrators can take. Different user profiles can be created with varying levels of access and permissions.
WordPress allows multi-factor authentication and Tainacan offers privacy tweaks that will protect files and metadata, but they need to be accompanied by a minimum set of backend configurations to protect against attacks. This is handled well in the current use cases by WordPress’s huge ecosystem of plugins and best practices in hosting services.
Why ABPA Chose It For Their Project
The Brazilian Association of Audiovisual Preservation (ABPA) created a collaborative repository of resources on audiovisual preservation in Portuguese, with more than 120 related items, including articles, debates, dissertations and theses, interviews, exhibitions, guides, books, and webinars, among others. Tainacan was chosen because of its versatility in supporting document types, in filtering the data, and in modes of presentation, but mostly because it allows associates to collaborate with new items and themes.
Spreadsheet
A spreadsheet is a simple tabular form that allows you to structure, format, sort, filter, locate, and efficiently disseminate data. It is a simpler way to organise and store information compared to a database software application. Spreadsheets can be used offline with programs like LibreOffice, Microsoft Excel, and Mac Numbers or online, with tools like Google Sheets, AirTable, and Microsoft Excel Web. Regardless of the app, a basic spreadsheet can help you systematise and control information.
Cost and Requirements
Costs vary. Spreadsheet tools like LibreOffice, Google Spreadsheet, and Numbers (Mac only) are free. Microsoft Excel is packaged with Microsoft 365, which is available through various pricing plans. Airtable is a “freemium” service that offers a no-cost basic tier and pricing plans for additional users and features.
Community Engagement and Collaboration
As a simple option, spreadsheets are more flexible but require more work to ensure consistency. For example, if the tool cannot enforce data entry rules, it becomes more important to employ a separate data dictionary and controlled lists of terms. Consider doing an assessment of the data consistency routinely when working with several collaborators. Periodically evaluate whether all contributors consistently fill the catalogue. Accidental changes to the data can be difficult to track, even with Google Sheets, which tracks changes but only if you look through the version history.
Usability and Accessibility
Spreadsheets are easy to use, but it’s essential to consider some best practices, like avoiding using the cell merge function, controlling data entry to enable effective indexing and filtering, and maintaining cataloging consistency. Also, configure the columns according to data type, such as a number, date or text.
While spreadsheets are more limited than other database applications, they typically include some useful features, like filters. You can use filters to quickly sort and limit the data you want to display (and Google Sheets offers the option to create a filter for you or all users).
Support and Training
Consider learning about the features of your spreadsheet software before starting your cataloging process to ensure it is sufficient for your needs. Since spreadsheet applications are widely used, they tend to have fairly good official documentation. It is also fairly easy to find third-party guidance online.
Scalability and Sustainability
Spreadsheets can become unwieldy when you have large quantities of data. You should be aware when the complexity of your data surpasses the capabilities of a spreadsheet. HURIDOCS shares useful advice on when to consider moving beyond spreadsheets and to database software, such as when you find yourself color-coding cells in the spreadsheet and doing little ‘hacks’ to find data; scrolling around a lot to find and edit information; or regularly reformatting data to fit the needs of different tools to make charts, maps or graphs.
Security, Privacy, and Sensitive Data
Online multi-user spreadsheets may be more practical for collaboration but are more insecure. You may opt to have only an offline version for security and privacy reasons, but maintaining consistency requires additional effort if a file is shared between contributors.
Why Laboratorio Popular de Medios Libres Chose Tainacan for the MemoriaVIVA Project
MemoriaVIVA is led by Laboratorio Popular de Medios Libres, in collaboration with Antena Negra TV, Sutty, Centro de Seguridad Informática y Tecnologías de la Comunicación, and with the support of WITNESS. Their mission is to collect multimedia material denouncing human rights violations, which is crucial in supporting lawyers, journalists, and civil society in justice processes. Each collection focuses on a specific period and territory, such as Peru, Argentina, and Chile, and the material is kept under strict security measures. The data about the videos, audio, and photographs submitted to the platform is collected via LimeSurvey, a free and open-source online survey web app. Access to the data is via spreadsheets exported from LimeSurvey that are analyzed/reviewed in LibreOffice. This process imposes challenges, such as losing information because of different formatting in each platform and the connection between the material (stored in a secure cloud) and its metadata. Because of that, the project is shifting to Tainacan.
—–
These are just four examples of cataloging tools WITNESS and partners have used in our projects. We hope that this post provides some insight into their relative strengths and weaknesses. What cataloging tool do you use, or are you thinking of using? If you are still deciding, we hope that this post can serve as a useful model for your own decision-making!
WITNESS has just launched the global campaign ‘#ArchiveLife: Preserving Collective Memory through Video Archiving’ – find out more here!
AUTHORS:
Yvonne Ng (she/her) is an audiovisual archivist with 15 years of experience working at the intersection of human rights, video and technology, and archives. As Senior Program Manager of Archives at WITNESS, she collaborates with regional teams to support partners to preserve human rights video, and develops accessible guidance and training materials related to archiving and preservation. Yvonne holds an M.A. in Moving Image Archiving and Preservation from NYU.
Ines Aisengart Menezes (she/her) is the Archives Program Coordinator and Technical Director of the ABPA – Brazilian Association of Audiovisual Preservation. Ines holds an M.A. in Preservation and Presentation of the Moving Image at the University of Amsterdam – UvA.
~ Published on 14th May 2024.