INTRODUCTION

CAREFULLY curated open datasets of music are indispensable resources for research. In various information retrieval domains (such as computer vision), significant advances were possible after the release of large open datasets. Such resources promote the reproducibility of research and ensure quality control (hence improvement). They also improve access by enabling researchers without the means to collect large datasets to contribute to research in these fields.

Digital artifacts of music accompanied with metadata and expert annotations is a basic need for data-driven approaches used in Music Information Retrieval (MIR) and computational musicology (Serra, 2017). There is a scarcity of open music recording collections for music research, primarily due to copyright issues. Some of the common ways the music research community has been tackling the problem of open datasets have been through the sharing of automatically computed audio features instead of the original recordings (Bertin-Mahieux, 2011; Porter et al., 2015) or providing researchers access to machines that can run analyses on local servers where data is stored. These limited sharing opportunities provide important means for reproducibility while facilitating collaborative work. However, the full potential (as observed in domains like computer vision) is far from being achieved due to the inaccessibility of the actual data itself, severely limiting research to problems that utilize available features. In addition, recent developments in deep learning technologies have led to the development of end-to-end systems for music applications, where the input is the audio recording.

Serra (2014) has defined the principles to be taken into consideration for the creation of new research corpora, which can also serve for evaluating the goodness of the data collection for a particular research task. According to such principles, the purpose of the corpus has to be explicit, the corpora must have good data coverage for the phenomenon under study, the data must be as complete as possible and of good quality, and finally, corpora should be reusable, implying that the data should be available for other researchers. Wilkinson et al. (2016) recently formulated a set of guidelines to improve the findability, accessibility, interoperability, and reuse (FAIR) of digital assets. 5 While Serra (2014) emphasized the quality and coverage of data, the FAIR principles emphasize machine-actionability because humans increasingly rely on computational support to deal with data as a result of the increase in volume, complexity, and creation speed of data. We consider the tenets and guiding principles by Serra (2014) to build research corpora with good quality and coverage while emphasizing FAIR principles to store, navigate and reuse data. In addition, we wish to provide open access to audio, metadata, and annotations, while allowing for the datasets to grow both in size and quality through community-driven efforts.

Tzanetakis (2014) observed an increasing interest in expanding data-driven research in MIR to music traditions around the world not commonly researched in this discipline, which poses hitherto unknown challenges and interesting research directions to improve the current state of the art. A major effort in that direction was the CompMusic 6 project focused on developing culture-specific analysis methods for various music cultures (Serra, 2011). The project provided an impetus to data-driven MIR in those music cultures, opening up new research directions, problems, and challenges. A primary effort of the project was to build research corpora and datasets for MIR in those music cultures. These corpora 7 have been created to study particular music traditions and include audio recordings plus complementary information that describes the recordings.

Srinivasamurthy et al. (2014) presented various datasets 8 for Indian Art Music (IAM), collected within the CompMusic project. Porter et al. (2013) initially presented a system named Dunya 9 for browsing audio music collections in the music cultures studied in CompMusic and an Application Programming Interface (API) for accessing metadata and audio files. Dunya has evolved to comprise the music corpora and related software tools that have been developed as part of the CompMusic project. The set of open-source software tools PyCompMusic 10 can be used to organize, access, and analyze these corpora. Each corpus has specific characteristics, and the developed software tools allow us to process the available information to study and explore the characteristics of each musical repertoire.

While metadata and annotations from the CompMusic datasets could be shared openly, we do not have the rights to share all the audio recordings since the music is copyrighted. Although the recordings are publicly accessible (through commercial albums), copyright still restricts many forms of usage that might be necessary for computational research. In addition, annotations could be performed only for parts of the audio, which limited their potential to be used in data-driven musicological studies.

In this paper, we present Saraga Indian Art music (Carnatic and Hindustani) open data collections, consisting of audio recordings and corresponding time-aligned melody, rhythm, and structural annotations for the complete duration of the recordings. The audio in Saraga collections comprises a subset of Dunya collections curated within the CompMusic project. The Carnatic collection includes recordings of live concerts, with a part of the collection available in multitrack. The Hindustani collection contains compilations by several renowned artists. The audio data spans about 96 hours of music accompanied by editorial metadata and time-aligned structural, melodic, and rhythmic annotations. We have obtained permission to share these audio recordings for research purposes under Creative Commons (CC) licenses 11, and the annotations are publicly available. The audio, metadata, and annotations are accessible through an API. A companion website 12 for the Saraga collection provides Python notebooks and example code snippets to access and explore the collections.

The datasets are intended to be useful for different MIR tasks in Indian art music and statistical computational musicology. The audio recordings and concerts were chosen to reflect the current state of music and performance practices. The availability of data, annotations, and code under open licenses helps create standardized open datasets for different MIR tasks in Indian art music, such as predominant melody extraction, meter tracking, structural segmentation, pattern similarity, pattern detection, and pattern discovery. An API to store/access data and annotation allows for effective indexing, sharing, and reusing the datasets.

The audio with rich metadata and annotations can help bootstrap datasets for relevant research problems posed in the analysis of Indian art music. These annotated collections are also useful to explore supervised machine learning methods for analysis. The annotations act as ground truth to build and improve automatic extractors for different melodic and rhythmic descriptors. Multitrack audio recordings are rarely available in MIR datasets. Hence, they are valuable resources when performing additional tasks that might be difficult or impossible in stereo mixes. For example, predominant melody extraction is much easier when a monophonic vocal track is available, as opposed to a stereo mix with another melodic instrument, and onset detection of percussion strokes will likely work better on drum tracks. The availability of multitrack audio in the Saraga Carnatic music collection is the first of its kind in IAM and will enable novel research directions such as melody extraction, source separation, automatic mixing, and performance analysis. The well-curated editorial metadata in the collections are useful for semantic analysis of different entities in editorial metadata, such as artists, compositions, forms, and relationships between them. The size of the collections allows the inference of statistically valid insights into music and aid in computational musicology. In addition, the collections can be useful to music listeners by providing an enriched music listening experience.

The Saraga collections were conceived to be a living collection that continues to evolve in current socio-cultural contexts and grow in quantity, quality, and comprehensiveness of metadata and annotations. To facilitate this goal, we provide a mechanism for the research/music community to contribute to the collections with more audio, manual annotations, or algorithmic extractors for automatic annotations. It is envisioned that Saraga datasets are adopted by the community at large for MIR with Indian Art Music.

INDIAN ART MUSIC

In the context of this paper, IAM refers to two art music traditions of the Indian subcontinent: Carnatic music, widespread in the southern regions of the Indian subcontinent (South India and Sri Lanka), and Hindustani music (or North Indian Classical music), prominent in the northern and central regions of India, Pakistan, Nepal, Afghanistan, and Bangladesh. IAM today is a confluence resulting from cultural interactions between the Persian, Greek, Arabic, Iranian and Indian cultures (Saraf, 2011).

While Hindustani music has a higher impact from influences external to the Indian subcontinent, Carnatic music is relatively insulated from those influences (Subramaniam, 1999). Both traditions are well studied with sophisticated and grounded music theory. IAM has a large audience, continues to evolve in the current socio-cultural context, and has attracted high interest from music scholarship. Different aspects of IAM history, theory, practice, and performance have been studied and documented in musicological and popular literature in both English and Indian languages (e.g., works by Bhatkhande, 1984, 1990; Sambamoorthy, 1998; Viswanathan et al., 2004; Trivedi, 2008; Raja, 2012; Ramanathan, 1999; Meer et al., 1980; Jha; 2001; Deshpande, 1989; Danielou, 2010; Krishna, 2013 provide a non-exhaustive sampling of current and past literature). The presence of a large, dedicated audience and research literature forms a solid basis for studying these music cultures from both musicological and computational perspectives.

Both Hindustani and Carnatic music is performance-oriented, heterophonic, with the main melody being sung or played by the lead artist and strongly improvisatory in nature. Vocal music is dominant in both traditions but more in Carnatic music. A typical arrangement in a performance of IAM consists of a lead performer, a rhythm accompaniment (typically mridangam and tabla in Carnatic and Hindustani music, respectively), a constant sounding drone in the background and an instrumental melodic accompaniment. In both traditions, rāga is the melodic framework, and tāla is the rhythmic framework. Rāga (melodic framework) and tāla (rhythmic framework) concepts have been largely discussed in a multitude of studies and will not be reviewed in detail here (Bagchee, 1998; Sambamoorthy, 1998), but it is emphasized that these concepts are fundamental in the analysis of Indian art music.

IAM is taught orally through a lineage of teachers and students. While a skeletal prescriptive notation based on an Indian solfège (called the sargam) exists, the notation is not standardized and mainly serves as a mnemonic aid for melodies and lyrics and is not directly used during a concert performance. The improvisatory nature and the absence of descriptive music scores are significant contributing factors to the recognizable stylistic differences that exist between different interpretations of a composition, melody, or a musical concept.

Ganguli and Rao (2018) summarize the rāga as being somewhere between a scale and tune, providing a grammar that specifies tonal material, tonal hierarchy, and a set of characteristic melodic phrases. Every rāga has a set of characteristic melodic phrases that act as building blocks to construct melodies and provide a base for artists to express their creativity through improvisation within the rāga grammar. A typical concert starts with the rendition of characteristic phrases of the rāga being performed. Musicians and musicologists often consider that a rāga can only be learned by getting familiar with landmark compositions and hence, the typical phrases in the rāga. Characteristic melodic phrases are also the most prominent cues used by human listeners for identifying rāgas (Krishna & Ishwar, 2012). Annotated characteristic melodic phrases in an audio recording of IAM is hence an important descriptor for melodic analysis.

Rhythmic organization in IAM relies on the tāla framework, which comprises fixed-length hierarchical time cycles. The instant of the beginning of each tāla cycle (also the end of the previous) is referred to as sama (or sam in Hindustani music), and it is highly significant structurally, marking time boundaries of important melodic and rhythmic events. The sam frequently marks the coming together of the rhythmic streams of soloist and accompanist and the resolution point for rhythmic tension (Clayton 2008, p. 81). Sama annotations are hence valuable for rhythmic and metrical analysis of IAM recordings.

DATA COLLECTION AND CONTENT

Many types of musically relevant digital artifacts are used in MIR. The most commonly used ones are music scores, digital audio recordings of musical performances, and music-related texts. Given that an audio recording of a music piece is the central element of music consumption, distribution, and description, the Saraga collections are organized in the same fashion. The meta-information for the audio recording is often as important as the recording itself since it helps to build interlinking relationships between the recording, musical concepts, and musicians. The collections hence comprise albums/concerts with audio recordings, along with the accompanying metadata and annotations for each recording. Music-related texts or scores are not included.

Hindustani music is spread over a wider geographical region. As a result, it is much more diverse and heterogeneous than Carnatic music. Several forms and styles exist, among which the khyal form is the most frequently performed. The Saraga dataset includes Hindustani recordings in the khyal form. In Carnatic music, a concert is the natural unit of music performance (which is also the preferred unit for organization and digital distribution). Carnatic music concerts typically comprise around ten music pieces involving performances based on compositions. The Carnatic collection of the Saraga dataset is also organized in a similar fashion. While both the music traditions have a rich instrumental repertoire, Saraga collections focus on and comprise only vocal performances.

Saraga collections have been designed under the principles outlined by Serra (2014). Throughout the CompMusic project, several challenges and opportunities were identified in MIR for Indian art music, which provided indicators of necessary metadata and annotations useful for computational research. Some of the most relevant annotations have been carried out with the purpose of developing datasets for a variety of well-defined melodic and rhythmic analysis tasks targeted in the CompMusic project. While the Saraga collections are envisioned to grow in both quality and quantity through community contribution, the current state of the collection includes carefully curated content described in the following subsections.

ORGANIZATION AND ACCESS

The organization of the Saraga collections is based on the tenets of easy access for both human and machine consumption, open availability of data and code, reproducibility, easy adoption by research, musician and music listener community, and enabling contribution by the community to grow the dataset in quality, quantity, coverage, and completeness.

Table 1. The content and coverage statistics of the Saraga collections

Content	Hindustani collection	Carnatic collection
Total releases	36	26
Total recordings	108	249
Total artists (lead and accompanying)	36	64
Compositions	113	202
Unique rāga	61	96
Unique tāla	9	10
Total duration	43.6 hours	52.7 hours

The contents of the collections can be accessed through an open-source API for machine consumption. The API allows access to audio, metadata stored in MusicBrainz, and manual/automatic annotations. Alternatively, a publicly available repository with a complete dump of metadata and annotations is also available to enable versioning and community contribution. The extractors used to derive automatic annotations are available through PyCompMusic tools.

APPLICATIONS

In addition to the MIR and computational musicology research problems described in previous sections that could potentially use the Saraga collections, we present other applications using the collections. We present two examples that use Saraga collections for different applications in music education, understanding, exploration, and discovery.

SUMMARY

Recently, there have been many contributions in developing open datasets for music analysis and exploration. However, most resources created are restricted in one way or another. Datasets containing annotations often refer to inaccessible audio, or when audio is available, they are often short snippets that fail to represent a real-world scenario. The scarcity of large and well-annotated open music collections continues to hinder reproducible research.

With this paper, we described the largest open-access annotated data collections for Indian Art Music. We also provide a companion website and example scripts to facilitate access and use of the data by other researchers. The audio in the datasets are representative of current performance practices and are released under Creative Commons, enabling reproducible research with the audio data. Further, a part of the Carnatic music collection is available in multitrack and provides a rare opportunity for researchers to analyze interactions between the tracks while also enabling more accurate analyses and extraction of melodic and rhythmic descriptors.

The annotations provided in the Saraga datasets have the potential to serve many research tasks in MIR and computational musicology. For example, the melodic and rhythmic annotations in the datasets are particularly useful for studies focusing on meter tracking, structural segmentation, pattern similarity, pattern detection, and pattern discovery tasks. These tasks are amongst the most popular and relevant in the MIR domain.

Metadata stored in an open encyclopedia such as MusicBrainz in linked form is of great use to study relationships such as stylistic similarities, time evolution of specific characteristics (using date information), and accessing other linked resources (such as Wikipedia) to build knowledge bases (Koduri, 2016). Open metadata is also preferred for enabling the extension of its size through collaborative efforts of the MusicBrainz community. Apart from artist and album information, the metadata we have curated involves tags for rāga, tāla, and form. This categorical information is a basic need in most computational studies but yet, hardly accessible in most other corpora.

The audio recordings in the collections are organized into source and derived files, with version control and easy access through the PyCompMusic API and a git repository. The git repository allows for a complete, up-to-date view of the data collection and provides version control for audio, metadata, manual and automatic annotations. Envisioned as a living and growing collection, we have provided mechanisms to contribute open audio and annotations to the collections. Both the research and music community are encouraged to use the datasets and contribute to it with more audio, better manual annotations, and algorithms for extraction of automatic annotations.

Other than research applications, we demonstrated two use cases that utilize the datasets for music understanding and exploration. Musical Bridges provides interactive tools for understanding and appreciating Hindustani music. The Saraga app is a mobile application for music exploration that streams audio and visualizes the annotations to help users view specific instances of particular aspects for IAM (e.g., typical melodic phrases of a rāga). The Saraga app can be used by music listeners for an enhanced experience when interacting with the Saraga music collections.

Open datasets are important for conducting collaborative work between different disciplines as they serve as resources collected collaboratively, adding different layers of annotations and information from different disciplines to enrich the data. Such open datasets are one of the needs of interdisciplinary fields of music analysis where annotations/labels by musicologists, musicians, and engineers are required to build large databases indispensable for data-driven knowledge extraction. We sincerely hope researchers from many different disciplines will use the Saraga collections to trigger collaborative interdisciplinary studies.