Martin Ivanov, Author at ServiceNow Guru

Custom queue event handling in ServiceNow – Implementation steps

Martin Ivanov — Tue, 29 Oct 2024 09:57:11 +0000

Background

Looking at the ServiceNow native processes, one can easily see that big portion of them are event-based, rather than synchronous. This is especially true for the processes which are not critical to the user experience or the ones which are not dependencies of other business logic.

In a nutshell, an event is logged in a queue and when system resources are available, the event is picked up and processed by the associated Script Action.

Below is a simple visual representation of the process along with explanation (source: Steven Bell) :

0. I register my new event in the Registry, create my Script Action associated to that event, and if needed my Script Include which could be called by the Script Action. Registering my event tells the Worker to listen for that event, and that it will be expected to do something with it.

1. Something executes a gs.eventQueue statement which writes an event record on the queue. BTW, this is not an exhaustive list.

2,3,4. The event worker(s), whose job it is to listen for events listed in the Registry, picks up the event and sees if there is a Script Action(s) associated with the registered event.

5,6. If a Script Action is found to run then it is executed which in turn may execute my Script Include if I choose.

Remember the info message when adding a role to a user or a group:

What’s happening behind – an event is logged to the queue and the roles are added to the group in the first possible moment, when the system has resources for that. Usually this is near real-time, but in case of higher priority operations are already queued, this will wait till they free up some processing power.

Now if one is implementing an application, based on synchronous logic, occupying almost all the system resources, this may lead to performance implications, slowing down the instance tremendously.

One possible approach in such cases is to shift from synchronous processing to event-based processing, which will lead to better performance.

But since events are being logged (unless another queue is explicitly specified) to the default queue, we might run into performance issues again.

Here comes the custom queue implementation. It is nothing more than a separate queue to which events can be queued explicitly, leveraging the fifth parameter of gs.eventQueue() API (more on that later).

Implementation

The implementation process is similar with the normal event-based logic implementation. We need to have:

An event registered in the event registry
A Business rule or any other logic to fire the event
A Script action to process the event
Custom queue processor

I will not discuss the first three, because these are pretty straightforward, and docs are easily available.

Custom queue processor implementation

The easiest way to create a processor for a custom queue is to:

go to System Scheduler -> Scheduled Jobs -> Scheduled Jobs
find a job with Trigger type = interval (i.e. ‘text index events process’)

change the name (it can be anything) replace ‘text_index’ with the name of your custom queue inside the fcScriptName=javascript\:GlideEventManager().process(); line
set Next action to be in the near future, i.e. 30 seconds from the current moment (This is very important in order to get the job running)

(optional) edit the Repeat interval (short repeat interval may have some negative impact on the system performance, but at the same time, the lower the repeat interval, the sooner your event will be queued and processed)
Right click -> Insert and stay! Do not Save/Update!

You can have one or more custom queues, depending on the purpose. These must be aligned with the system resources – nodes, semaphores, workers. I will not go deeper on these, more information can be found in the Resources chapter below.

Logging an event to a specific (custom) queue

gs.eventQuque() API accepts 5 parameters:

Event name
GlideRecord object
Param1
Param2
(optional) queue

This fifth optional parameter ‘queue’ is the one that tells the system to which event queue an event should be logged.

We can log an event to the custom queue we have created above (‘custom_queue_one’) we can use the following line of code:

gs.eventQueue('event.name', grSomeObject, null, null, ‘custom_queue_one’);

NB: queue name (fifth parameter) must be exactly the same as the one we’ve passed to the GlideEventManager during the process creation above.

Everything else (script actions, etc. is the same like in a normal event logging)

Good practices

Just because you can, doesn’t mean you should – this implementation is applicable only to cases where huge amounts of records must be processed (see Performance chapter)
Naming is important, give your names and processor readable names
For optimal performance, multiple custom queues can be created to handle a particular event. In this case, the event logging must be done in a way that ensures even distribution between the queues. To better organize these, one possible approach can be to:
- Create a script include, holding an array with the names of your event queues
- Use the following line of code to randomly distribute events to the queues:

gs.eventQueue('event.name', grSomeObject, null, null, event_queues[Math.floor(Math.random()*event_queues.length)]);

where event_queues is an array containing the names of your queues

Performance

Even though we implement this approach to achieve performance, for low number of transactions it does not yield any performance gain, because of the Repeat interval – the longer it is, the slower will be the overall wait time
For large number of transactions (thousands of records), the achieved performance gain can be really significant. In one of my implementations, I was able to achieve 30x faster execution.

More information

The post Custom queue event handling in ServiceNow – Implementation steps appeared first on ServiceNow Guru.

The C4 model for ServiceNow Solution Design and Documentation

Martin Ivanov — Mon, 17 Jun 2024 16:24:25 +0000

Having been in the software development space for over seven years, I have observed multiple issues occurring in almost every project, regardless of industry, company size, type of implementation, tools, etc. You might have guessed already: I am talking about documentation, or rather the lack thereof.

Problem definition

Even when documentation is present, it is often written in a freeform manner, comprising a mixture of screenshots, URLs, and explanations. Each stream lead or architect has their own understanding of best practices and documentation layout, which, especially in large and complex implementations, results in many different documents that are inconsistently structured and formatted.

These documents might be readable during the implementation phase or shortly thereafter, but if a new joiner needs to quickly catch up, they will have a really hard time—trust me, I’ve been there and done that.

The list of issues with freeform documentation goes on and on. Sometimes an executive or an enterprise architect needs to quickly check something and make a decision based on it. Instead, they must read through the aforementioned documents and spend a lot of time trying to find that one piece of information. And we all know how frustrated they become when their most precious resource is wasted. We all do.

Here comes the solution: The C4 model.

Quick overview of the C4 model

The C4 model was created by the Software Architect Simon Brown between 2006 and 2011, but it became popular after the launch of an official website and an article in 2018.

According to the official website, “The C4 model was created as a way to help software development teams describe and communicate software architecture, both during up-front design sessions and when retrospectively documenting an existing codebase. It’s a way to create maps of your code, at various levels of detail, in the same way you would use something like Google Maps to zoom in and out of an area you are interested in.”

Further clarified in Wikipedia, “The C4 model documents the architecture of a software system, by showing multiple points of view that explain the decomposition of a system into containers and components, the relationship between these elements, and, where appropriate, the relation with its users.

The viewpoints are organized according to their hierarchical level:

Context diagrams (level 1): show the system in scope and its relationship with users and other systems;
Container diagrams (level 2): decompose a system into interrelated containers. A container represents an application or a data store;
Component diagrams (level 3): decompose containers into interrelated components, and relate the components to other containers or other systems;
Code diagrams (level 4): provide additional details about the design of the architectural elements that can be mapped to code. The C4 model relies at this level on existing notations such as Unified Modelling Language (UML), Entity Relation Diagrams (ERD)or diagrams generated by Integrated Development Environments (IDE).

For level 1 to 3, the C4 model uses 5 basic diagramming elements: persons, software systems, containers, components and relationships. The technique is not prescriptive for the layout, shape, color and style of these elements. Instead, the C4 model recommends using simple diagrams based on nested boxes in order to facilitate interactive collaborative drawing. The technique also promotes good modelling practices such as providing a title and legend on every diagram, and clear unambiguous labelling in order to facilitate the understanding by the intended audience.”

Different levels of zoom allow you to tell different stories to different audiences. (Source: c4model.com)

The C4 model for ServiceNow Solution Architecture design and documentation

I’ve been using the C4 model for documenting ServiceNow solutions ever since I joined the architecture team I am currently working on. This kind of documentation is standard practice on every project we work on, along with a few others.

In the following paragraphs, I will go through each of the four levels with a simple example of an integration between ServiceNow and an external system, feeding foundation data into the platform.

Here is a helicopter view of the whole diagram. Do not be frustrated if you don’t understand it the first time. Bear with me.

Helicopter view of an example integration, documented according to the C4 model

Let’s break it down

Level 1: System Context Diagram

As stated previously, the goal of the System Context Diagram is to document the main system and its interactions with users and other systems. In our case, this is ServiceNow, which interacts with BMC Remedy, feeding foundation data. The main persona here is the Foundation Data Admin on the Remedy side, who is responsible for the data’s completeness and correctness.

Zoom-in: System Context Diagram

Level 2: Container Diagram

The documentation states that this level aims to decompose a system into interrelated containers. A container represents an application or a data store.

In the current example, we can easily see that the foundation data import is implemented through the system scheduler, which executes data imports via REST API calls to the external system, using a technical account for authentication. The data is transformed by Transform Maps and stored in the database.

Zoom-in: Container Diagram

Level 3: Component Diagram

Zooming in to the Component layer, we try to decompose containers into interrelated components and relate the components to other containers or systems.

We’ve mentioned the system scheduler and the data source at the Container level, but here at the Component layer, we can get further information about the specific scheduled job, its name and interval, the name of the data source, as well as more specific details about the REST message and the service account.

Zoom-in: Component Diagram

Level 4: Code

The goal of this layer is to provide additional details about the design of the architectural elements that can be mapped to code. In most cases, this can be skipped or partially documented for the sake of simplicity. In our practice, we try to document at least the custom code that we have implemented as part of the project.

Looking back at my simple example, at the Code level, we can see a breakdown of the Data Source code and the Script Include that it references.

Zoom-in: Code

Benefits

While it might look a bit confusing when seen for the first time, an implementation documented according to the C4 model has multiple benefits.

First and foremost, it is documentation readable by all personas – executives like CIOs and COOs, Enterprise and Solution Architects, Business Process Consultants, and of course, implementers and developers. A picture is worth a thousand words, right?

When this type of documentation becomes a standard practice, it becomes really easy to onboard new people to a project or communicate design decisions with various stakeholders because they already know how to read it.

On the other hand, documenting every implementation with the same standard approach quickly builds up a library of reusable reference architectures that can be used with just a few tweaks or as communication/demonstration material during the early project phases while decisions are being made.

Last but not least, frameworks like Mermaid.js allow for diagrams to be presented as code, giving us a chance to store these in centralized repositories. Furthermore, we can use AI to analyze them and help us continuously improve our solution architecture.

Recap

The C4 model, an “abstraction-first” approach to diagramming software architecture, can be applied for the design and documentation of ServiceNow solutions, introducing a new standard approach for documentation with multiple benefits for both customers and implementation partners.

The post The C4 model for ServiceNow Solution Design and Documentation appeared first on ServiceNow Guru.