Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger Activation Failure Post-Selective Deployment #386

Open
Gexdobro opened this issue Feb 29, 2024 · 8 comments
Open

Trigger Activation Failure Post-Selective Deployment #386

Gexdobro opened this issue Feb 29, 2024 · 8 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@Gexdobro
Copy link

Gexdobro commented Feb 29, 2024

Describe the bug
Hi Kamil!

We are using the PowerShell module for our selective ADF objects deployments, and it works great (ADF as a resource pre-exists). Basically, we defined our PowerShell script to explicitly include the desired objects to be deployed from our repository. During deployment, the only caveat is that after the first deployment (creation) of each trigger, it stays disabled, even though in the trigger JSON definition it is declared as Started (desired state). I am attaching the parameters set for a deployment below.

The described case is "S01" from your matrix-of-behavior-logic-1.

Parameters the script has been run with:

Includes                       : {[pipeline.example_1, AdfObject], [trigger.example_1, ], [linkedService.example_1, ],[pipeline.example_2, AdfObject]…}
Excludes                       : {}
DeleteNotInSource              : True
StopStartTriggers              : True
CreateNewInstance              : False
DeployGlobalParams             : True
FailsWhenConfigItemNotFound    : True
FailsWhenPathNotFound          : True
IgnoreLackOfReferencedObject   : False
DoNotStopStartExcludedTriggers : False
DoNotDeleteExcludedObjects     : True
IncrementalDeployment          : False
TriggerStopMethod              : DeployableOnly
TriggerStartMethod             : BasedOnSourceCode

To Reproduce
Generate a PowerShell script with parameters mirroring the ones provided above. Execute the script to initiate a selective deployment.

Expected behavior
Trigger is active after first selective deployment.

Version of PowerShell module of:
azure.datafactory.tools 1.8.0

Logs

The deployment log erroneously indicates that the trigger has already started, despite it not being active in reality.

===================================================================================
STEP: Starting triggers...
TriggerStartMethod = BasedOnSourceCode
- Trigger already started: example_1

Trigger definition

{
	"name": "example_1",
	"properties": {
		"annotations": [],
		"runtimeState": "Started",
		"pipelines": [
			{
				"pipelineReference": {
					"referenceName": "example_1",
					"type": "PipelineReference"
				}
			}
		],
		"type": "ScheduleTrigger",
		"typeProperties": {
			"recurrence": {
				"frequency": "Month",
				"interval": 12,
				"startTime": "2024-01-15T10:54:00Z",
				"timeZone": "UTC",
				"schedule": {
					"minutes": [
						0
					],
					"hours": [
						0
					],
					"monthDays": [
						1
					]
				}
			}
		}
	}
}

@Gexdobro Gexdobro added the triage Issue is under investigation label Feb 29, 2024
@NowinskiK
Copy link
Member

Are you sure that the trigger is NOT Started after the deployment?
Can you double check that with the following code?

Get-AzDataFactoryV2Trigger -ResourceGroupName "ADF" -DataFactoryName "WikiADF"

@NowinskiK NowinskiK self-assigned this Feb 29, 2024
@Gexdobro
Copy link
Author

Thank you for your quick answer!

Yes, I am certain these triggers stays "Stopped" deployment regardless of the runtimeState value in the JSON definition trigger file, at least with the presented above parameters. I have double checked the post-deployment status in the portal and also using the PowerShell command you suggested.

@NowinskiK
Copy link
Member

It must be some specific scenario in your case because this is a very common pattern to deploy and start a trigger.
This normally works for hundreds if not thousands of users (including Unit Test), otherwise I would get this issue here immediately.
Can you proof this doesn't work within some video or script so I could easily run & reproduce the issue?

@Gexdobro
Copy link
Author

Gexdobro commented Feb 29, 2024

At the moment I can only share the script below unfortunately:

param
(
    [parameter(Mandatory = $true)] [String] $rootFolder,
    [parameter(Mandatory = $true)] [String] $dataFactoryName,
    [parameter(Mandatory = $true)] [String] $resourceGroupName,
    [parameter(Mandatory = $true)] [String] $location
)

$adf = Import-AdfFromFolder -RootFolder "$rootFolder" -FactoryName $dataFactoryName
$opt = New-AdfPublishOption

# Trigger:
$opt.Includes.Add("trigger.trigger1", "")

# Other objects
$opt.Includes.Add("integrationruntime.*", "")
$opt.Includes.Add("factory.*", "")
$opt.Includes.Add("managedVirtualNetwork.*", "")

$opt.CreateNewInstance = $false
$opt.DeleteNotInSource = $true
$opt.TriggerStopMethod = "DeployableOnly"

Publish-AdfV2FromJson -RootFolder "$rootFolder" -ResourceGroupName "$resourceGroupName" -DataFactoryName "$dataFactoryName" -Location "$location" -Option $opt

A trigger json definition can be as the one described in the previous comment.

I believe a critical factor that might be contributing to this problem is the parameter $opt.TriggerStopMethod being set to "DeployableOnly" - but for a selective deployment this value is desired. Here is a very similar lead but in this case trigger is defined as already deployed (second run):

#369

@Gexdobro
Copy link
Author

Gexdobro commented Mar 5, 2024

Hi @NowinskiK - will you be able to help with this issue?

@NowinskiK
Copy link
Member

I have limited capacity atm, but I will take a look at this as soon as I can.

@Gexdobro
Copy link
Author

Gexdobro commented Mar 6, 2024

Thank you very much! Your assistance would be greatly appreciated!

@NowinskiK NowinskiK added bug Something isn't working and removed triage Issue is under investigation labels Mar 10, 2024
@NowinskiK NowinskiK added this to the v1.10 milestone Mar 10, 2024
@NowinskiK
Copy link
Member

This only happens when TriggerStopMethod = 'DeployableOnly'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants