"All Country" and "All Topic" Databases

JewishGen ShtetlMaster integration protocol

· "All Country" Databases
    · Historical Perspective
    · Regional Databases
    · Overlapping Regions
· "All Topic" Databases
· Data Placement
· Examples

"All Country" Databases

Historical perspective

The earliest genealogical-related records of value to Jewish researchers generally date back to the end of the 18th century.  Since that time, the political boundaries of Europe (principally of Central and Eastern Europe) have undergone many changes.  Thus, genealogists must undertake research with an understanding of the political history of the towns and areas of interest.

For example, the city now known as "Lviv, Ukraine", was formerly known as "Lemberg, Galicia, Austria" before World War I.  Between the world wars it was known as "Lwów, Poland".  After WWII it was "L'vov, U.S.S.R.", and since the breakup of the former Soviet Union, it has been known as "Lviv, Ukraine".  Thus this city has been in Galicia (Austrian Empire), Poland, the Soviet Union, and Ukraine, at various points in time.

In the early years of Internet on-line research, when there were few databases available, checking them all was a simple and relatively fast process.  However, as the interest in Jewish genealogy has grown and as databases housed on JewishGen have multiplied, finding and searching all the available information has become time-consuming, and keeping up-to-date with newly added data in all the databases has become very difficult.

Meeting the needs of researchers in the growing world of JewishGen databases

The mission of JewishGen is to enhance the ability of all researchers to find relevant data by providing easily accessible information.  To accomplish this objective, JewishGen has — in consultation with both its in-house and independent content providers — developed a "Regional Database" strategy to handle the problems of overlapping areas/borders.  This initiative is to prevent any single database provider from denying ease of accessibility to researchers and to ensure that all data related to a specific region will be available in a one-time search of that region's database.

Regional Databases – the solution

To ensure the success of the Regional Database program, JewishGen and its content providers have established the following principles:

  1. Data applicable to one or more regions is made available in each applicable region's "All Country" database.
  2. Each dataset has a "primary source" tag to identify the individual / project / organization under which it was created. All web pages, search results and any other information provided by the JewishGen's aggregated databases (an "All Country" or "All Topic" database) shall clearly identify all data from each "primary source", with full clear attribution given to each dataset's creator/contributor.
  3. The "All Country" databases and their management shall have no rights to the primary source data, other than to display it in their database as long as they are housed on JewishGen.
  4. Database content providers shall be consulted on all matters relating to sharing of data but shall not have the rights to deny sharing of such data where it is deemed appropriate by the JewishGen Database Sharing Committee. (See "Data Placement", below).
  5. Full rights to any enhancements to the primary data, created by a aggregated database (i.e. an "All Country" or "All Topic" database), shall accrue to the primary source.  Example: adding notes to entries, extending fields in entries (extracts), or any improvement of any kind.

JewishGen's current "All Country" databases are:

Database Name Country(s) Database URL
JewishGen Belarus Database Belarus http://www.jewishgen.org/databases/Belarus
JewishGen Scandinavia Database Denmark, Sweden, Norway, Finland http://www.jewishgen.org/databases/Scandinavia
JewishGen Germany Database Germany http://www.jewishgen.org/databases/Germany
JewishGen Hungary Database Hungary, Slovakia, Croatia http://www.jewishgen.org/databases/Hungary
JewishGen Latvia Database Latvia, Estonia http://www.jewishgen.org/databases/Latvia
JewishGen Lithuania Database Lithuania http://www.jewishgen.org/databases/Lithuania
JewishGen Poland Database Poland http://www.jewishgen.org/databases/Poland
JewishGen Romania Database Romania, Moldova http://www.jewishgen.org/databases/Romania
JewishGen Ukraine Database Ukraine http://www.jewishgen.org/databases/Ukraine
JewishGen UK Database United Kingdom http://www.jewishgen.org/databases/UK
JewishGen USA Database United States http://www.jewishgen.org/databases/USA

Other "All Country" databases are planned (e.g. "All Israel", "All Austria-Czech", "All South Africa"), as a significant quantity of data for those regions becomes available, and respective database managers for each come forward.  There are also specific political, linguistic and technical issues for each region.

Note that while the "All Belarus" and "All Latvia" databases are JewishGen databases, operated by JewishGen; JRI-Poland and the LitvakSIG's "All Lithuania Database" databases are operated by independent organizations hosted by JewishGen.  However, this fact is not relevant to the database integration protocol — all data will be made available to researchers via the appropriate "All Country" search interface.

For example, if a dataset relevant to Lithuania is donated to JewishGen, then that data will be made available to researchers via the "JewishGen Lithuania Database" interface, as per item #1 above.  (And possibly other interfaces, such as a neighboring region's "All Country" database, or an "All Topic" database (see below), as determined by the JewishGen Database Sharing Committee).  However, the LitvakSIG will not have any rights to this data (as per item #3 above).  The contributor of this dataset retains all the rights, and their copyright, introduction/description and attribution will appear in all instances of their data (as per item #2 above).

For a current example of how this attribution works, see the JewishGen Latvia Database, which contains data contributed by two different organizations (the JewishGen Latvia SIG, and the Courland Research Group), as well as by other individuals.  Each dataset is properly attributed to its original donors/authors/compilers, wherever that data appears in any interface.

Overlapping regions

Each locality in ShtetlMaster will belong to exactly one non-overlapping "Region", for multi-database integration purposes.

Consider a map of Europe and mark on it the various countries/regions that have existed over time.  There is significant overlap, due to shifting historical borders.  Think of each marked region as having its own unique color, and each overlapping area also having its own unique color.  We need to distinguish each uniquely colored area. There will be about two dozen distinct regions.

These non-overlapping regions, as I have defined them, are:

# Region "All Country" Database(s) to include in Historical Notes Example cities
1 Congress Poland POLAND* The ten gubernias of Russian Poland (1815-1918), minus northern Suwalki. Warszawa, Plock, Siedlce, Lublin, Kielce, Radom, Kalisz, Piotrków, Lomza, Suwalk
2 West Galicia POLAND*, GALICIA In Austrian Empire, 1772-1917; in Poland 1918+ Kraków
3 East Galicia POLAND, GALICIA, UKRAINE* In Austrian Empire 1772-1917; in Poland 1918-1939; in Ukraine 1945+ Lviv, Ivano-Frankivsk, Tarnopol
4 Eastern Belarus BELARUS* That part of modern Belarus that was not incorporated into inter-war Poland. (Former eastern Minsk, southern Vitebsk, Mogilev gubernias). Gomel, Minsk, Mogilev, Bobruisk
5 Western Belarus BELARUS*, POLAND The parts of modern Belarus that were in Poland between the wars. (Former eastern Grodno, western Minsk gubernias). Grodno, Brest, Pinsk
6 Northern Suwalki LITHUANIA*, POLAND The northern districts of Suwalk gubernia. In Russian Poland before WWI, now in Lithuania. Mariampol, Kalwarya, Vladislavov, Volkovisk
7 Western Volhynia POLAND, UKRAINE* In Russian Empire before WWI, in Poland between the wars, in Ukraine now. Luck, Kremenets, Rovno
8 Russian Ukraine UKRAINE* Eastern Ukraine, former Russian Empire gubernias. Excludes parts that were in Poland or Czechoslovakia between the wars. Kiev, Zhytomir, Chernigov, Kherson, Odessa, Dnipropetrovsk
9 Subcarpathia HUNGARY, CZECH, UKRAINE* Area that was in Hungary pre-WWI, in Czechoslovakia between the wars, and is in Ukraine today. Uzhgorod
10 Posen / Prussia POLAND*, GERMANY In Prussia before WWI, in Poland thereafter. Poznan, Breslau
11 Kovno / North Vilna LITHUANIA* Kovno gubernia, plus the northwestern uyezds of Vilna gubernia. All now in Lithuania. Vilnius, Kaunas
12 South Vilna LITHUANIA, POLAND, BELARUS* The southern uyezds of Vilna gubernia. In Russian Empire before WWI, in Poland between the wars, now in Belarus. Lida, Oshmiany, Vilieka, Disna
13 Latvia LATVIA* Livland, Courland, and western Vitebsk gubernias Riga, Mitau
14 Hungary HUNGARY* ... to be defined ... Budapest
15 Austria AUSTRIA* ... to be defined ... Vienna
16 Bohemia / Moravia CZECH*, AUSTRIA All of modern Czech republic, former Austrian provinces. Prague, Brno
17 Slovakia CZECH, HUNGARY All of modern Slovakia, former northern counties of Hungary. Bratislava, Kosice
18 Germany GERMANY* Modern Germany... TBD Berlin
19 Bessarabia ROMANIA In Russian Empire 1812-1856, 1878-1918; Part of Romania 1856-1878, 1918-1940; In USSR (Moldavian SSR) 1940-1991. Kishinev
20 Transylvania ROMANIA*, HUNGARY Province of Austro-Hungarian Empire 1867-1918; part of Romania following WWI. Cluj
21 North Bukovina UKRAINE*, ROMANIA, AUSTRIA Province of Austro-Hungarian Empire 1775-1918; became part of Romania 1918-1944. After WWII, became part of USSR; today in Ukraine. Cernovtsy
22 South Bukovina ROMANIA*, AUSTRIA Province of Austro-Hungarian Empire 1775-1918; became part of Romania after WWI. Suceava
23 Romania ROMANIA* ... to be defined ... Wallachia, Moldavia ... some overlap with Hungary Bucharest, Iasi

The list of names in capital letters in the second column above indicates which "All Country" database(s) that region would be included in.  An asterisk (*) following a name in the second column indicates the political entity where that region is located today.  Some regions will be included in more than one "All Country" database.  For example, data for the city of Mariampol and the surrounding area (Northern Suwalki) is included in both the "JewishGen Poland Database" and "JewishGen Lithuania Database", because this area was in both Poland and Lithuania, at various points in time.

However, we will draw some arbitrary lines, to avoid absurd amounts of overlap.  Namely, for inter-war Poland — which included nearly all of Grodno, Vilna, and Volhynia gubernias, as well as parts of Minsk gubernia — we will not include all Vilna and Minsk area data in the "JewishGen Poland Database", unless the data is part of a larger inter-war dataset that otherwise qualifies for inclusion in that database (i.e. a nation-wide dataset, such as the 1929 Polish Business Directory).

This also breaks down rather nicely by gubernia (and we can use other historical jurisdictions as pseudo-gubernias for non-Czarist regions), so the "gubernia" field in ShtetlMaster helps place things in the right regional database.

Of course there are a handful of historical anomalies — short-lived territories, small border disputes, tiny regions that changed hands multiple times, etc. — that can't easily be accommodated by the regional system, and will not be considered.  We can't do everything without the system becoming ridiculously complex.

"All Topic" Databases

In addition to the "All Country" databases, there will also be overlapping "All Topic" databases, which cut across geographic lines.  "All Topic" databases combine similar record types, regardless of locality.  This adds a third dimension to the "All Country" overlapping regions.

Examples of "All Topic" databases include:

All data contributed to any of the above databases will also automatically be included in the proper JewishGen "All Country" database (if such database currently exists), and vice-versa.

For example, all data for Latvian cemeteries contributed to the JOWBR will also be searchable via the "JewishGen Latvia Database"; all burials in Belarus will be searchable via the "JewishGen Belarus Database", etc.  Conversely, any cemetery/burial data contributed to the "JewishGen Belarus Database" will also be made available to researchers via the JOWBR.

Another example: The names in the JewishGen Yizkor Book Necrology Database from Yizkor Books for Polish towns will be searchable via the "JewishGen Poland Database" interface; names from Yizkor Books for Lithuanian towns will become searchable via the "JewishGen Lithuania Database", etc.  See more examples below.

Data Placement

The JewishGen Database Sharing Committee will determine in which "All Country" and/or "All Topic" database(s) each contributed dataset will be placed — in consultation with the database contributor and the respective database managers.

The primary criteria for data placement is what will be most convenient and useful for the researchers — the end users of the databases.  The users are concerned only with accessibility; not with who created/indexed/donated the data.

To use an analogy: When a book is donated to or acquired by a library, it goes on a shelf with other books on the same topic — for the convenience of the library patrons — as determined by the librarian, in consultation with standard cataloging rules.  A book about Lithuania is placed next to other books about Lithuania — without regard to the author/publisher.  In the library catalog, the book is still clearly indicated as being by a particular author and publisher, and has all of the donor's information — but its shelf location is determined purely by its subject matter, since that's what the patrons really care about.  A patron looking for information about Lithuania will naturally search under "Lithuania", not under the name of a book's author, publisher, or donor.  It is the book's content that is of primary concern, and the same holds true for databases.

Examples:

This data sharing protocol is currently implemented in the Yizkor Book Necrology Database and the All Vsia Rossiia Database (both are "All Topic" databases) and the JewishGen Belarus Database (an "All Country" database).  Here are two current examples of how this works:

Here are some additional hypothetical examples.  The diagram below represents four hypothetical datasets (represented by circles), contributed by various groups and individuals, and four "All Country" / "All Topic" database searches (represented by rectangular buttons) which allow access to this data:

  1. "Kraków Cemetery", donated by JRI-Poland.  It is searchable via both the "JewishGen Poland Database", and via the "JewishGen Online Worldwide Burial Registry" (JOWBR) database, because Kraków is in Poland, and because this is cemetery data, respectively.
  2. "Mariampol Births", donated by user ABC.  It is searchable via both "JewishGen Poland Database" interface, and via the "JewishGen Lithuania Database" interface, because Mariampol was in Poland, and is now in Lithuania. (Region 6, Northern Suwalki).
  3. "Bialystok Deportations", donated by user DEFG.  It is searchable via both "JewishGen Poland Database", and via the "JewishGen Holocaust Database" interface, because Bialystok is Poland, and this is data relating to the Holocaust.
  4. "Kovno Ghetto Burials", donated by user XYZ.  It is searchable via three interfaces: the "JewishGen Lithuania Database"; and via the "JewishGen Holocaust Database"; and via the "JewishGen Online Worldwide Burial Registry" (JOWBR) — because Kovno is Lithuania, this is data relating to the Holocaust, and this is burial data, respectively.


Warren Blatt,
August, 2000
Updated Mar 3 2002, Jul 30 2003.