Package com.linkedin.venice.helix

Class HelixReadOnlySchemaRepository

java.lang.Object

com.linkedin.venice.helix.HelixReadOnlySchemaRepository

All Implemented Interfaces:

ReadOnlySchemaRepository, StoreDataChangedListener, VeniceResource

Direct Known Subclasses:

HelixReadOnlyZKSharedSchemaRepository

public class HelixReadOnlySchemaRepository
extends Object
implements ReadOnlySchemaRepository, StoreDataChangedListener

This class is used to cache store schema and provide various query operations. This expected user is router to support venice client schema query operations, and storage node for schema validation; TODO: 1. This class needs to periodically clear/warm up local cache since we might miss some notifications: 1.` Re-connect; One way to achieve to record the previous clear timestamp, and clear it again when the duration exceeds some amount of time in query functions. We need to reach out Helix team for this issue since it will impact Helix cache as well.

Field Summary

Fields

Modifier and Type	Field	Description
static final int	VALUE_SCHEMA_STARTING_ID

Constructor Summary

Constructors

Constructor	Description
HelixReadOnlySchemaRepository(ReadOnlyStoreRepository storeRepository, org.apache.helix.zookeeper.impl.client.ZkClient zkClient, HelixAdapterSerializer adapter, String clusterName, int refreshAttemptsForZkReconnect, long refreshIntervalForZkReconnectInMs)

Method Summary

Modifier and Type	Method	Description
void	clear()	Clear local cache and watches
DerivedSchemaEntry	getDerivedSchema(String storeName, int valueSchemaId, int derivedSchemaId)
GeneratedSchemaID	getDerivedSchemaId(String storeName, String derivedSchemaStr)	Look up derived schema id and its corresponding value schema id by given store name and derived schema.
Collection<DerivedSchemaEntry>	getDerivedSchemas(String storeName)
SchemaEntry	getKeySchema(String storeName)	This function is used to retrieve key schema for the given store.
DerivedSchemaEntry	getLatestDerivedSchema(String storeName, int valueSchemaId)	Get the most recent derived schema added to the given store and value schema id
RmdSchemaEntry	getReplicationMetadataSchema(String storeName, int valueSchemaId, int replicationMetadataVersionId)
Collection<RmdSchemaEntry>	getReplicationMetadataSchemas(String storeName)
SchemaEntry	getSupersetOrLatestValueSchema(String storeName)	Caller shouldn't modify the returned SchemeEntry
SchemaEntry	getSupersetSchema(String storeName)	Get the superset value schema for a given store.
SchemaEntry	getValueSchema(String storeName, int id)	This function is used to retrieve the value schema for the given store and value schema id.
int	getValueSchemaId(String storeName, String valueSchemaStr)	This function is used to retrieve value schema id for the given store and schema.
Collection<SchemaEntry>	getValueSchemas(String storeName)	This function is used to retrieve all the value schemas for the given store.
void	handleStoreChanged(Store store)
void	handleStoreCreated(Store store)	Do nothing here, since we want to warm up local cache gradually.
void	handleStoreDeleted(String storeName)	For store deletion, we need to delete the local cache entry right way, otherwise the local cache may contain the stale entries for store-delete-and-add scenario.
boolean	hasValueSchema(String storeName, int id)	This function is used to check whether the value schema id is valid in the given store.
void	refresh()	Refer to clear() This function will clear the local cache/watches, and populates the schemaMap from schemaRepository.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.venice.meta.ReadOnlySchemaRepository

getLatestDerivedSchema

Methods inherited from interface com.linkedin.venice.meta.StoreDataChangedListener

handleStoreDeleted

Field Details

VALUE_SCHEMA_STARTING_ID

public static final int VALUE_SCHEMA_STARTING_ID

See Also:

• Constant Field Values

Constructor Details

HelixReadOnlySchemaRepository

public HelixReadOnlySchemaRepository(ReadOnlyStoreRepository storeRepository,
 org.apache.helix.zookeeper.impl.client.ZkClient zkClient,
 HelixAdapterSerializer adapter,
 String clusterName,
 int refreshAttemptsForZkReconnect,
 long refreshIntervalForZkReconnectInMs)

Method Details

getKeySchema

public SchemaEntry getKeySchema(String storeName)

This function is used to retrieve key schema for the given store. If store doesn't exist, this function will return null; If key schema for the given store doesn't exist, will return null; Otherwise, it will return the key schema; Caller shouldn't modify the returned SchemeEntry

Specified by:

getKeySchema in interface ReadOnlySchemaRepository

Returns:

null, if key schema for the given store doesn't exist; key schema entry, otherwise;

getValueSchema

public SchemaEntry getValueSchema(String storeName,
 int id)

This function is used to retrieve the value schema for the given store and value schema id. Caller shouldn't modify the returned SchemeEntry

Specified by:

getValueSchema in interface ReadOnlySchemaRepository

Returns:

null, if the schema doesn't exist; value schema entry, otherwise;

hasValueSchema

public boolean hasValueSchema(String storeName,
 int id)

This function is used to check whether the value schema id is valid in the given store.

Specified by:

hasValueSchema in interface ReadOnlySchemaRepository

getValueSchemaId

public int getValueSchemaId(String storeName,
 String valueSchemaStr)

This function is used to retrieve value schema id for the given store and schema.

Specified by:

getValueSchemaId in interface ReadOnlySchemaRepository

Returns:

SchemaData.INVALID_VALUE_SCHEMA_ID, if the schema doesn't exist in the given store; schema id (int), if the schema exists in the given store

getDerivedSchemaId

public GeneratedSchemaID getDerivedSchemaId(String storeName,
 String derivedSchemaStr)

Description copied from interface: ReadOnlySchemaRepository

Look up derived schema id and its corresponding value schema id by given store name and derived schema. This is likely used by clients that write to Venice

Specified by:

getDerivedSchemaId in interface ReadOnlySchemaRepository

Returns:

a pair where the first value is value schema id and the second value is derived schema id

getDerivedSchema

public DerivedSchemaEntry getDerivedSchema(String storeName,
 int valueSchemaId,
 int derivedSchemaId)

Specified by:

getDerivedSchema in interface ReadOnlySchemaRepository

getValueSchemas

public Collection<SchemaEntry> getValueSchemas(String storeName)

This function is used to retrieve all the value schemas for the given store. Caller shouldn't modify the returned SchemeEntry list.

Specified by:

getValueSchemas in interface ReadOnlySchemaRepository

Returns:

value schema list

getDerivedSchemas

public Collection<DerivedSchemaEntry> getDerivedSchemas(String storeName)

Specified by:

getDerivedSchemas in interface ReadOnlySchemaRepository

getSupersetOrLatestValueSchema

public SchemaEntry getSupersetOrLatestValueSchema(String storeName)

Caller shouldn't modify the returned SchemeEntry

Specified by:

getSupersetOrLatestValueSchema in interface ReadOnlySchemaRepository

getSupersetSchema

public SchemaEntry getSupersetSchema(String storeName)

Description copied from interface: ReadOnlySchemaRepository

Get the superset value schema for a given store. Each store has at most one active superset schema. Specifically a store must have some features enabled (e.g. read compute, write compute) to have a superset value schema which evolves as new value schemas are added.

Specified by:

getSupersetSchema in interface ReadOnlySchemaRepository

Returns:

Superset value schema or null if store does not have any superset value schema.

getLatestDerivedSchema

public DerivedSchemaEntry getLatestDerivedSchema(String storeName,
 int valueSchemaId)

Description copied from interface: ReadOnlySchemaRepository

Get the most recent derived schema added to the given store and value schema id

Specified by:

getLatestDerivedSchema in interface ReadOnlySchemaRepository

getReplicationMetadataSchema

public RmdSchemaEntry getReplicationMetadataSchema(String storeName,
 int valueSchemaId,
 int replicationMetadataVersionId)

Specified by:

getReplicationMetadataSchema in interface ReadOnlySchemaRepository

getReplicationMetadataSchemas

public Collection<RmdSchemaEntry> getReplicationMetadataSchemas(String storeName)

Specified by:

getReplicationMetadataSchemas in interface ReadOnlySchemaRepository

refresh

public void refresh()

Refer to clear() This function will clear the local cache/watches, and populates the schemaMap from schemaRepository.

Specified by:

refresh in interface VeniceResource

clear

public void clear()

Clear local cache and watches

Specified by:

clear in interface VeniceResource

handleStoreCreated

public void handleStoreCreated(Store store)

Do nothing here, since we want to warm up local cache gradually.

Specified by:

handleStoreCreated in interface StoreDataChangedListener

handleStoreDeleted

public void handleStoreDeleted(String storeName)

For store deletion, we need to delete the local cache entry right way, otherwise the local cache may contain the stale entries for store-delete-and-add scenario.

Specified by:

handleStoreDeleted in interface StoreDataChangedListener

handleStoreChanged

public void handleStoreChanged(Store store)

Specified by:

handleStoreChanged in interface StoreDataChangedListener