Managing Physical File Storage

Former Member — Tue, 04 Aug 2020 21:55:57 GMT

Current Revision posted to Developer Training by Former Member on 08/04/2020 21:55:57

The Verint Community platform's [[Centralized File Storage|Centralized File Storage System]] abstracts physical file storage from [[Interacting With Files|logical file storage]] and includes support for physically storing files on the operation system's file system or Amazon S3. Additional physical file storage providers can be added to support storing files elsewhere.

[toc]

Why Should I Create a New File Storage Provider?

A new file storage provider should only be required to physically store files on a new physical technology or service. Verint Community includes support for operating system file storage (local hard drive or UNC share) and Amazon S3. If another storage system, for example, a different cloud storage provider must be used, a new file storage provider can be used and configured to be used by [[Centralized File Storage|the centralized file system]].

File Storage Provider Considerations

When developing new file storage providers, it is important to ensure that [[Centralized File Storage|CFS]] expectations are supported. Key considerations include:

The provider implementation must have a public, parameterless constructor.
File store keys must be between 1 and 255 characters (each character must be supported by the Windows file system)
Paths must be between 0 and 769 characters (each character must be supported by the Windows file system)
File names must be between 1 and 255 characters (each character must be supported by the Windows file system)
Paths should be considered as prefixes, not delimited "folders" when searching for sub-folders.
Paths created with the AddPath() method are expected to be returned by GetPaths() calls even if the path contains no files.
The provider is responsible for removing unused paths if a physical representation of the path (without respect to files) is required by the underlying physical implementation.
The provider is responsible for checking permissions on secured file stores ([[api-documentation:ISecuredCentralizedFileStore Plugin Type|ISecuredFileStore]] and [[api-documentation:IGloballySecuredCentralizedFileStore Plugin Type|IGloballySecuredFileStore]]) before serving files (if the provider serves files directly).
The provider is responsible for interacting with "findable" file stores when a file cannot be found when serving file contents ([[api-documentation:IFindableCentralizedFileStore Plugin Type|IFindableCentralizedFileStore]]).
The provider is responsible for providing an implementation of [[api-documentation:ICentralizedFile Plugin Supplementary Type|ICentralizedFile]] for files returned by the provider which includes a full implementation, including a custom, direct HTTP handler for the GetDownloadUrl() result.
Any [[api-documentation:ICentralizedFile Plugin Supplementary Type|ICentralizedFile]] instances returned must exist in underlying storage. If the files does not exist, a null value must be returned.

Provider Interface Types

File storage providers are defined using the [[api-documentation:ICentralizedFileStorageProvider Provider Type|ICentralizedFileStorageProvider]] interface. A new instance of the implementation will be created for each CFS file store configured to use the provider. For each file store, the provider will be instantiated and the instance will be notified of the file store it represents via the Initialize() method, which provides the file store key and the XML node defined in the communityserver.config file which can be used to provide configuration information to the provider. The members required by ICentralizedFileStorageProvider should be self-descriptive with respect to the considerations listed above.

Individual providers can expose additional/optional functionality by implementing additional extension interfaces to ICentralizedFileStorageProvider:

Exposing Events

All file storage providers should support events, but events are technically optional. To expose file manipulation events, the provider should implement [[api-documentation:IEventEnabledCentralizedFileStorageProvider Provider Type|IEventEnabledCentralizedFileStorageProvider]]. This interface adds a set-only property, EventExecutor, to receive the [[api-documentation:ICentralizedFileEventExecutor Plugin Supplementary Type|ICentralizedFileEventExecutor]] implementation from the Verint Community platform which can be used by the provider to fire CFS-related events.

When firing events from a custom provider, ensure that when a global change is made that event parameters that don't apply are set to null and not an empty string. For example, when a complete file store is deleted, the OnBeforeDelete() and OnAfterDelete() parameters for path and fileName should be null, because the deletion affects all paths and files.

Direct HTTP Handling

By default the generic download URL for CFS files will issue an HTTP redirect to the provider specific download URL for a file (via the provider's implementation of [[api-documentation:ICentralizedFile Plugin Supplementary Type|ICentralizedFile.GetDownloadUrl()]]). Extensions to ICentralizedFileStorageProvider enable direct HTTP handling without a client-based redirect which can improve end-user performance.

[[api-documentation:IHttpRenderableCentralizedFileStorageProvider Provider Type|IHttpRenderableCentralizedFileStorageProvider]] extends ICentralizedFileStorageProvider and adds the HandleHttpRequest() method. This method is called by the generic CFS HTTP handler and allows the custom provider to handle the existing request directly. If the method returns true, the platform assumes the request is fully handled. If the provider returns false, the default behavior will be processed likely resulting in an HTTP redirect.

Persistent URL Generation

Persistent URLs to CFS-stored files are embedded in content that is dynamically generated, but cached for a potentially long time, such as HTML output from a widget or processed LESS files. By default, persistent URLs generated by the CFS use the generic URL format. If a custom provider can provide a long-term URL (one that is not time-bombed), it can identify this capability to the Verint Community platform by implementing the [[api-documentation:IPersistentUrlGeneratingFileStorageProvider Provider Type|IPersistentUrlGeneratingFileStorageProvider]] interface and implementing the GetPersistentDownloadUrl() method.

When a file storage provider supports persistent URLs, the provider's persistent URL will be used instead of the generic URL when a persistent URL is requested for a CFS-stored file.

Configuration

The mapping of file storage providers to file stores is defined within the communityserver.config file. To use a custom provider, this configuration will need to be updated. See installation and configuration documentation to update the configuration of the CFS appropriately.

Tags: files, CFS, Centralized File Storage

Managing Physical File Storage

Ben Tiedt — Thu, 13 Jun 2019 19:28:56 GMT

Revision 1 posted to Developer Training by Ben Tiedt on 06/13/2019 19:28:56

The Telligent Community platform's [[Centralized File Storage|Centralized File Storage System]] abstracts physical file storage from [[Interacting With Files|logical file storage]] and includes support for physically storing files on the operation system's file system or Amazon S3. Additional physical file storage providers can be added to support storing files elsewhere.

[toc]

Why Should I Create a New File Storage Provider?

A new file storage provider should only be required to physically store files on a new physical technology or service. Telligent Community includes support for operating system file storage (local hard drive or UNC share) and Amazon S3. If another storage system, for example, a different cloud storage provider must be used, a new file storage provider can be used and configured to be used by [[Centralized File Storage|the centralized file system]].

File Storage Provider Considerations

When developing new file storage providers, it is important to ensure that [[Centralized File Storage|CFS]] expectations are supported. Key considerations include:

The provider implementation must have a public, parameterless constructor.
File store keys must be between 1 and 255 characters (each character must be supported by the Windows file system)
Paths must be between 0 and 769 characters (each character must be supported by the Windows file system)
File names must be between 1 and 255 characters (each character must be supported by the Windows file system)
Paths should be considered as prefixes, not delimited "folders" when searching for sub-folders.
Paths created with the AddPath() method are expected to be returned by GetPaths() calls even if the path contains no files.
The provider is responsible for removing unused paths if a physical representation of the path (without respect to files) is required by the underlying physical implementation.
The provider is responsible for checking permissions on secured file stores ([[api-documentation:ISecuredCentralizedFileStore Plugin Type|ISecuredFileStore]] and [[api-documentation:IGloballySecuredCentralizedFileStore Plugin Type|IGloballySecuredFileStore]]) before serving files (if the provider serves files directly).
The provider is responsible for interacting with "findable" file stores when a file cannot be found when serving file contents ([[api-documentation:IFindableCentralizedFileStore Plugin Type|IFindableCentralizedFileStore]]).
The provider is responsible for providing an implementation of [[api-documentation:ICentralizedFile Plugin Supplementary Type|ICentralizedFile]] for files returned by the provider which includes a full implementation, including a custom, direct HTTP handler for the GetDownloadUrl() result.
Any [[api-documentation:ICentralizedFile Plugin Supplementary Type|ICentralizedFile]] instances returned must exist in underlying storage. If the files does not exist, a null value must be returned.

Provider Interface Types

Individual providers can expose additional/optional functionality by implementing additional extension interfaces to ICentralizedFileStorageProvider:

Exposing Events

All file storage providers should support events, but events are technically optional. To expose file manipulation events, the provider should implement [[api-documentation:IEventEnabledCentralizedFileStorageProvider Provider Type|IEventEnabledCentralizedFileStorageProvider]]. This interface adds a set-only property, EventExecutor, to receive the [[api-documentation:ICentralizedFileEventExecutor Plugin Supplementary Type|ICentralizedFileEventExecutor]] implementation from the Telligent Community platform which can be used by the provider to fire CFS-related events.

Direct HTTP Handling

Persistent URL Generation

Persistent URLs to CFS-stored files are embedded in content that is dynamically generated, but cached for a potentially long time, such as HTML output from a widget or processed LESS files. By default, persistent URLs generated by the CFS use the generic URL format. If a custom provider can provide a long-term URL (one that is not time-bombed), it can identify this capability to the Telligent Community platform by implementing the [[api-documentation:IPersistentUrlGeneratingFileStorageProvider Provider Type|IPersistentUrlGeneratingFileStorageProvider]] interface and implementing the GetPersistentDownloadUrl() method.

When a file storage provider supports persistent URLs, the provider's persistent URL will be used instead of the generic URL when a persistent URL is requested for a CFS-stored file.