Deleting multiple articles via admin panel

I have an article application containing many entries, over 30k and want to delete them all, in the admin panel I am allowed to delete all but it only seems to delete 100 entries

If I delete the application the content is not removed, the application is hidden but still exists in the platform, is there a way to remove it and all its content ? 

  • Also seeing a few exceptions re timeout

    System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
       at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
       at System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error)
       at System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()
       at System.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()
       at System.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()
       at System.Data.SqlClient.TdsParserStateObject.TryReadByte(Byte& value)
       at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Telligent.Evolution.Data.ContentSqlDataProvider.Delete(Nullable`1 containerId, Nullable`1 applicationId)
       at Telligent.Evolution.Api.Content.ContentService.Delete(DeleteContentQuery query)
       at Telligent.Evolution.Api.Content.ApplicationStateChanges.Deleted(Guid applicationId)
       at Telligent.Evolution.Articles.Plugins.ArticleCollectionsApplicationType.Events_AfterDelete(ArticleCollectionAfterDeleteEventArgs e)
       at Telligent.Evolution.Components.EventsBase.Execute[T](Object eventObject, Action`1 executor, Boolean throwExceptions)
    ClientConnectionId:63bd8786-d74e-4fde-80b7-28d8a88ec6c8
    Error Number:-2,State:0,Class:11

    UnknownException: Exception occured while running background job (Telligent.Evolution.CoreServices.Tags.DeleteTagsJob, Telligent.E (Id=cb5a471a-2779-4a1d-82f7-06f3cdbcd0af, JobId=440db2fc-2976-413f-bd48-ba8597b9b566, Platform=evolution)  ExecutionTimeUtc: 10/08/2021 09:48:18Data:
    	ApplicationId : c3aa8f60-811a-4207-8559-92da001c2cb1
    	DYNAMIC_JOB_TYPE : Telligent.Evolution.CoreServices.Tags.DeleteTagsJob, Telligent.Evolution.Platform
    ). (An error occurred while executing a background job. The error has been logged for the administrator to review.) ---> System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
       at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
       at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Telligent.Evolution.Data.ProfiledSqlCommand.ExecuteNonQuery()
       at Telligent.Evolution.Data.TagSqlDataProvider.DeleteAll(TaggedContentDeleteQuery query)
       at Telligent.Evolution.CoreServices.Tags.Implementations.TagService.DeleteAll(TaggedContentDeleteQuery query)
       at Telligent.Evolution.CoreServices.Tags.DeleteTagsJob.Execute(JobData jobData)
       at Telligent.Evolution.Components.Jobs.DynamicEvolutionJobRunner.SubExecute(JobExecutionContext context)
       at Telligent.Evolution.Components.Jobs.BaseJobRunner.Execu
  • Articles follows a "soft delete" pattern, where the content is still available for a certain period of time in case it needs to be "undeleted"/restored. The Expunge Deleted Articles job controls how long soft deleted articles are retained.

    Deleting 100 at a time via the management panel would be the way to do it via the UI; you could also create code to use the API (REST, widget script, or in-process) to page through and delete articles.

    The timeouts you are seeing are due to various core services attempting to process the large amount of changes from the application delete. I've filed a bug to investigate better handling of those cases of very large amounts of data. The workaround would be the code solution I mentioned above.

    TE-17030: Timeouts processing ArticleCollection delete with very large number of Articles

    Planned

  • So to clarify is there no way to remove these deleted applications ? 

  • They are expunged in the same job (Expunge Deleted Articles). You can adjust the settings of that job to have them delete faster (e.g., lower number of days to keep, and potentially increasing the job run schedule temporarily if you'd like them gone immediately).

  • Ah yes some script in widget sandbox should be the quickest way - thanks

  • Hi Stephen

    I write an automation to purge these 36k articles and can see they are flagged as IsDeleted and appear in the deleted tab in article admin panel but are still not being removed?

    #set($articles = $articles_v1_articles.List("%{ ArticleCollectionId = $collectionId, IsDeleted = 'false', pageSize=100, PageIndex=0 }"))
    
    #foreach ($article in  $articles)
        $core_v2_eventLog.Write("Purging articles - $articles.Count - $articles.TotalCount - $collectionId - $article.Title", "%{ Category = 'IET Purge', EventType = 'Information' }")
        $articles_v1_articles.Delete($article.Id)
    #end


    I have reset the job to run every 15 minutes with a threshold of 1 day and still no joy

    The versions in the database do not appear to be flagged as IsDeleted but are also not flagged as IsActive

    Any suggestions as we currently have 750mb of table data we no longer need ? 

    The build is 12.0.2.17146

  • Hi Stephen 

    Further to this issue I ran a trace on the database and the lookup queries to locate articles to delete seemed to be running with no user id ? 

    If I manually re-ran the query from the trace it found zero records to process, but if I changed the user id passed in it worked fine 

    @SecureByUserId=0,@PermissionId='BBCD6087-5DCC-4E30-B5C2-808EE3E33AF8',@Fetch=100,@Offset=0

    e.g.

    @SecureByUserId=2100,@PermissionId='BBCD6087-5DCC-4E30-B5C2-808EE3E33AF8',@Fetch=100,@Offset=0

    I can supply the full queries if needed but didn't want to paste it publicly

  • Hi, there was an additional bug discovered, logged, and fixed for 12.0.3 release:

    TE-17035: Expunge Deleted Articles Job isn't actually expunging anything.

    Completed for 12.1, 12.0.3

  • Yes, this was the issue in the bug above :)

  • Verint | Telligent Community
    0 Verint | Telligent Community
    TE-17035: Expunge Deleted Articles Job isn't actually expunging anything. has been completed for 12.1, 12.0.3