An automated pipeline for Ultra-Violet Imaging Telescope

Abstract

We describe a versatile pipeline for processing the data collected by the Ultra-Violet Imaging Telescope (UVIT) on board Indian multi-wavelength astronomical satellite ASTROSAT. The UVIT instrument carries out simultaneous astronomical imaging through selected filters/gratings in far-ultra-violet (FUV), near-ultra-violet and visible (VIS) bands of the targeted circular sky field ($\sim 0.5^{\circ }$ dia). This pipeline converts the data (Level 1) emanating from UVIT in their raw primitive format supplemented by inputs from the spacecraft sub-systems into UV sky images (and slitless grating spectra) and associated products readily usable by astronomers (Level 2). The primary products include maps of Intensity (rate of photon arrival), error on Intensity and effective Exposure. The pipeline is open source, extensively user configurable with many selectable parameters and its execution is fully automated. The key ingredients of the pipeline include extraction of drift in the pointing of the spacecraft, and disturbances in pointing due to internal movements; application of various corrections to measured position in the detector for each photon – e.g., differential pointing with respect to a reference frame for shift and add operation, systematic effects and artefacts in the optics of the telescopes and detectors, exposure tracking on the sky, alignment of sky products from multi-episode exposures to generate a consolidated set and astrometry. Detailed logs of operations and intermediate products for every processing stage are accessible via user-selectable options. While large number of selectable parameters are available for the user, a well characterized ‘standard default’ set is used for executing this pipeline at the Payload Operation Centre (POC) for UVIT and selected products are archived and disseminated by the Indian Space Research Organization (ISRO) through its ISSDC portal.

Appendices

Appendix A. Modules used in the UVIT Level 2 Pipeline

This appendix provides complete technical details of the processing blocks used by the UVIT Level 2 Pipeline. Given the two different available modes of imaging operations the processing chains have distinct versions for each of these modes. There are several computational blocks with commonality in their function among the chains/versions. Hence, such processes are captured as Modules with selectable switches and parameters to meet the requirements of all similar applications (see Table 3 for a list of all Modules and their linkages to different chains). A complete list of switches and parameters for these Modules is available in Appendix K. The Modules are called by the chains and the resulting products from each Module follow a naming convention with name suffix embedded in them for ease of tracking as well as higher level of scripting for autonomous operations. Individual Modules are described next, in a natural order of their sequence of appearance in the chains for drift extraction: Relative Aspect (RA_INT and RA_PC) followed by those exclusively for the Sky Imaging (L2_PC and L2_INT). There are 20 Modules presented below in appendices numbered: A.1, A.2,$\ldots$ to A1.20. Every Module consists of several blocks operating in a sequence, and in general, each block operates on all the frames of an Episode before transferring control to the next block (with only exception of a few in the RA_INT chain).

1.1 Appendix A.1 DataIngest

The DataIngest is the very first block for any of the processing chains and it prepares all inputs data for processing in the subsequent Modules. Its key functionality is to import input data, check its integrity, sort them into various groups/tables and carry out certain computations. It directly interfaces with the ‘merged Level 1’ (mL1) data bundle which is the primary input to the pipeline, the Calibration Database, CAL_DB, as well as the user selected parameter set (Parameter Interface Library; PIL).

First, a brief description of the contents of input data bundle mL1 is presented. The mL1 contains all the Level 1 data products Science Data and Aux Data for the same Observation ID (ObsID; it includes Proposal ID, Target serial number and Pointing sequence number) in an organized manner. Generally the observations corresponding to a specific ObsID lasts multiple successive orbits. The directory structure of a representative dataset (mL1) is shown in Appendix D. Appendix E shows the directory structure of the Calibration Database, CAL_DB. The notation followed to represent multi-level directory structure in Appendix D as well as similar other Appendices E–J are as follows: a normal dash symbol represents files in current directory and a longer horizontal shift or dash represents the next lower sub-directory (followed by the sub-directory name) and the lines immediately following it give contents of that sub-directory. A few files with very long names are broken into two lines with the second line beginning with a large number of dots.

The Science Data are segregated into the three band wise directories. These directories in turn host multiple sub-directories, each of which correspond to one specific ‘Episode’ of data collection. Each sub-directory is populated by three files (FITS tables), the imaging data and two files (GTI and BTI) containing flags for a large number of parameters pertaining to the spacecraft and UVIT.

The imaging data are packed as a sequence of ‘frame’-s, one for each individual exposure, preserving their time order. Each frame comprises of an integral number of fixed size ‘packets’ (2048 bytes) whose contents and format depend on the imaging mode (INT/PC) configured. Each packet begins with the Synchronization Mark, Primary Header and a Secondary Header (includes an image frame counter and various other Counters with timing information) followed by 2016 bytes of image data and ending with a parity-code (CRC). The very first packet of every frame always holds the data for all settings for the Detector Module and the data for system status and health monitoring only. The second packet onwards contains image data. The total number of packets in any frame, $N_p$ depends on the imaging mode. The last packet for the frame usually needs padding to complete the stipulated fixed size (2048 bytes). In the case of Integration mode (INT), the raw pixel ADUs (2-bytes per pixel) are packed in the usual sequence of 2-D array covering the entire selected window region. Hence, the value of $N_p$ is always predictable for INT mode, e.g., maximum value of 262 corresponds to full $512\times 512$ window setting. On the other hand, in Photon Counting mode (PC), the raw pixel signals are processed on board to identify individual photon event and compute its centroid along two detector based axes (X, Y). After this processing, the data from the frame comprises of all extracted photon events packed as a sequence, with each photon event contributing six contiguous bytes. Accordingly, each packet can accommodate up to 336 photons. As a result, the value of $N_p$ in PC mode may vary from frame to frame, since it depends on the total number of photon events identified. The contents of the 6 bytes for every detected photon event are: centroid along X (2 bytes), centroid along Y (2 bytes), measures of asymmetry in the raw event footprint and local background (2 bytes).

The Auxiliary (Aux) data correspond to the entire time duration covering the full merged dataset. They are FITS tables containing uninterrupted continuous time series of various measured/derived parameters mostly originating in spacecraft sub-systems. Examples of Aux data that are used directly by the pipeline follow. The time correlation between the spacecraft’s clock (raw and calibrated) and internal clocks of UVIT (‘*.tct’). The attitude of the spacecraft: tabulated as quaternions representing rotation angles with respect to reference coordinate system – Roll, Yaw and Pitch (‘*.att’). The signals from the angular rate sensors (Gyros) for the three axes (Roll, Yaw and Pitch) which are tabulated separately (‘*.gyr’).

Figure 14 provides the sequence of operations carried out by DataIngest. DataIngest initially carries out the following functions: (i) import mL1 data, (ii) set up structured paths for Science Data and Aux Data, (iii) establish links to Calibration Database, (iv) import and interpret settings of various user selectable switches and values of parameters, from Parameter Interface Library, PIL, queries.

As a part of check on data integrity at a primitive level, the user selected action is carried out on individual data packets which failed the CRC test. Next, the headers of individual Science Data blocks corresponding to distinct Episodes are interpreted for various settings: respective observing Band, imaging Mode, Filter, the band selected for Master Clock. For each Episode, a time correlation between UVIT’s Master Clock and the absolute time (Universal Time Clock, UTC, stamped by Level 1 process) is constructed. Occasionally, the UTC shows erratic behaviour. Even when user selects to use the UVIT clock (i.e., with ‘utcFlag’ switch is OFF), a robust error tolerant scheme has been employed to establish a scheme capable of predicting Modified Julian Date (MJD) from UVIT clock, albeit with limited precision ($\sim$1 s).

Figure 14

Block diagram for DataIngest Module. This Module first collects all relevant input data from observations, calibration and user selections of configurable parameters. It checks integrity and quality of observational data and separates them bandwise in suitable formats (as per imaging mode: INT/PC) for processing by the later stages. It carries out time correlation and certain other functions as described in the text.

Next step deals with identification and treatment of corrupt and unusable data. The frame number and its corresponding time stamp must always increment in a regular fashion as per the selected parameters related to the frame read-out rate (with the exception of natural return to zero on overflow of the respective counters). This provides a good handle for identifying certain kinds of data corruption. The nature of unusual data patch includes the following occurrences noticed in most Level 1 datasets (in PC mode): mismatch between Frame Number and Frame Time (one of them being in the correct sequence and the other showing a JUMP ‘spike’, which can be positive or negative); completely discontinuous Frame Number and Frame Time inserted within normal good data sequence; data from a particular frame repeated elsewhere in the same dataset (i.e., for the same observing Episode); abrupt discontinuity in Frame Number which violates monotonicity, etc. These issues have come to light progressively during the mission and appropriate logic incorporated within this DataIngest Module. As a result this Module has evolved over longer time. The identified ‘outlier’ frames are discarded from further consideration. The fraction of input Level 1 data successfully translated by DataIngest Module to reorganized output usable for the subsequent Modules depends on frequency of occurrences of unusual data patches. Some statistical information based on past $\sim$4 years’ operation of UVIT are provided in Section 4.

Initial part of every Episode of observation includes data collected during ramping up of high voltages for detector safety, which is not scientifically useful since the gain is changing with time. These time stretched are identified and discarded. The primary time variable to be used extensively by the subsequent steps of the pipeline is generated by applying UTC correction to UVIT’s Master Clock (only if ‘utcFlag’ is selected to be ON). In the case the Good Time Interval (GTI) filtering is turned ON, the time stretches of data not meeting the criteria selected by the user are discarded. The precise value for exposure time per frame is computed and recorded. In actual processing so far, GTI filtering has never been used.

The next phase of processing in DataIngest depends on the imaging Mode. For INT, individual frames containing raw pixel counts are extracted and stored in separate 2-D FITS image files in a directory named ‘SignalFrames’ with nomenclature of each file having Frame number and Time stamp embedded in it. This directory also stores a list of extracted images.

For PC, sequence of individual frames with centroid coordinates in detector X–Y coordinate system and other details like Frame number, UVIT Master Clock time, UTC time, diagnostic measures of event’s raw $5\times 5$ pixel footprint – ‘Max–Min’ and ‘Min’ values of the four corners for every detected photon are all packed into a single ‘events’ file. It holds the data from the specific imaging Episode as one master list of photon events in chronological order as FITS table.

For both the Modes, the following products are included: UVIT-UTC time correlation table, log of data loss and an ‘info’ file for passing various computed and PIL parameters to the subsequent Modules.

Some long term effects of exposure to radiation (SEE) on the on board electronic components were noticed midway through the mission life. They were recoverable by hardware RESET to the system at periodic intervals. However, some fraction of the data collected were affected beyond the designed immunity in the pipeline. As a result additional software patches were developed and applied at a later stage to mitigate these effects. While nature of these fixes belongs to the DataIngest Module, they were accommodated in the envelope code for the chain (RA_INT) and described in Section 3.2.1.

A few general remarks follows, on the convention followed throughout the pipeline for flagging of unusual data/locations. First about flagging pixels/elements of a 2-D image/array, which need not be considered for processing due to either considered as ‘bad’ due to any defect or representing a region outside the active area of the detector. An unique value (−9999) is assigned to such pixels/elements for their easy recognition as an ‘invalid’ pixel/element by all processing blocks. No arithmetic operation is carried out on such flagged pixels/invalid elements and their values are left unaltered. On the other hand, the flagging of individual photons in the master event list are carried out by appending an additional column with an entry with a value ‘1’ ($=$good)/‘0’ ($=$bad) indicating their inclusion/exclusion for consideration in further processing (e.g., event located outside the detector’s active area; asymmetry in footprint signifying multi-photon event).

Directory structures of the products of DataIngest Module for INT and PC modes are provided in Appendix F and Appendix G, respectively.

1.2 Appendix A.2 uvtUnitConversion

Aim of this Module (called by all the versions of the chains) is to translate the image data into physical unit, making it independent of the frame integration time, selected for the observation. For INT mode data, first a dark frame, D(ix,jy), taken from CAL_DB is subtracted from the raw frame, RI(ix, jy), and then divided by the frame integration time, $T_i$, (earlier computed by DataIngest Module) making the pixel values to be ‘Signal/s’ (the code is designed to derive this dark frame by linear interpolation in time between dark frames taken before the sky-exposure and after the sky-exposure. But, in practice this dark frame is taken from the CAL_DB of ground calibration). This process is repeated for every image frame.

$$\texttt{I}\_\texttt{uc} (\texttt{ix},\texttt{jy})= [\texttt{RI}(\texttt{ix},\texttt{jy})-\texttt{D}(\texttt{ix},\texttt{jy})]/(\texttt{T}\_\texttt{i});$$

for all pixels (ix, jy) of the image, and repeated for all images; For PC mode data, each photon event is assigned a value of an event/s (one divided by the frame integration time) and recorded in an additional column of the master table of events. This column is referred to as Effective Number of Photons per second (ENP).

$$\texttt{ENP}\_\texttt{uc}(\texttt{k}) = [1/(\texttt{T}\_\texttt{i})];$$

for all events in master table ‘k’ $=$ 0 to ‘k_max’.

Note the suffix ‘uc’ which identifies the outputs with the Module. A similar naming convention is followed for all other Modules also.

1.3 Appendix A.3 uvtMaskBadPix

This Module is called by all versions of the processing chains. The functionality of this Module includes identifying and flagging of: (1) bad pixels in INT mode image arrays and (2) events in PC mode master table with centroids located in bad pixels as well as those satisfying criteria for being classified as multi-photon event. BAD_PIXELS files, for each of the three bands, are arranged as per the mode of observations and the chosen window size, in the CAL_DB. All the inactive pixels, either because of being outside the detector’s circular field of view or because of selection of one of the smaller window options, are represented as ‘bad pixels’ by populating ‘0’ entries (rest with ‘1’ entries), in the corresponding BAD_PIXELS file BP(ix, jy). For INT mode the flagging is effected on all bad pixels of each image frame.

$$\begin{aligned}\begin{array}{l} \texttt{if\,\,BP}(\texttt{ix}, \texttt{jy}) = \texttt{0},\,\,\texttt{then}\\ \texttt{I}\_\texttt{bp}(\texttt{ix}, \texttt{jy}) = \texttt{INVALID}\_\texttt{PIX}\_\texttt{VALUE};\\ \texttt{else}\, \texttt{I}\_\texttt{bp}(\texttt{ix}, \texttt{jy}) = \texttt{I}\_\texttt{uc}(\texttt{ix}, \texttt{jy});\end{array}\end{aligned}$$

As stated earlier (Appendix A.1), a value of ‘−9999’ is assigned for the ‘INVALID_PIX_VALUE’ flag. For PC mode, every photon event is assigned a value for the variable ‘BAD_FLAG’ depending on the integral parts of its centroid coordinates. The centroid coordinates, (x, y), are real numbers represented by an integer part (in the range: 0–511) and a fractional part. Consider the integer parts of the centroids (x, y) to be (ix, jy). A new column for ‘BAD_FLAG’ is appended to the master table of events, which is populated with

$$\texttt{BAD}\_\texttt{FLAG}(\texttt{k}) = \texttt{BP}(\texttt{ix},\texttt{jy}),$$

where centroid coordinates for the event ‘k’ are (x, y); for all events in master table ‘k’ $=$ 0 to ‘k_max’. This Module carries out another function for the PC mode, namely, flagging for multi-photon events. The option for discarding events having two neighbouring photons was introduced with the aim of achieving higher angular resolution. On board processing in PC, mode for detecting individual photon events from raw frames, also extract diagnostic information as stated earlier. The identification of a multi-photon event is based on this diagnostic data (‘Max–Min’ and ‘Min’ among the raw counts from four corners of the $5\times 5$ photon event footprint) and the selected threshold value, ‘mult_photon_thrsld’. For events meeting the above condition are flagged for potential rejection ($=$‘0’) in the ‘MULTI_PHOTON’ column appended in the master table of events.

$$\begin{aligned}\begin{array}{l} \texttt{if}\,\, [\texttt{Max}-\texttt{Min}](\texttt{k}) > \texttt{mult}\_\texttt{photon}\_\texttt{thrsld},\\ \texttt{then\,\,MULTI}\_\texttt{PHOTON}(\texttt{k}) = \texttt{0};\\ \texttt{else\,\,MULTI}\_\texttt{PHOTON}(\texttt{k}) = 1;\end{array}\end{aligned}$$

for all events in master list ‘k’ $=$ 0 to ‘k_max’. Subsequent processing has the option to discard every frame with even one event with multi-photon flag set for rejection from further consideration. However, this option has never been used.

1.4 Appendix A.4 uvtCosmicRayCorr

This Module handles the effects of cosmic rays and is called by all versions of the processing chains. There is a provision for by passing this Module.

This Module attempts to mitigate the effects of individual primary cosmic ray (CR) hits as well as shower of secondary charged particles generated by interaction of CR within and near the detector. Some of the charged particles would pass through the CMOS imager leading to a large signal in individual pixels, while other charged particles and photons would generate light pulses some of which look like photon events in the PC mode. In the INT mode, the former can be detected as unusually large signals in individual pixels the latter cannot be isolated as these would be similar to genuine photon events. In the PC mode, some of the hits would generate a large shower of events in a frame which can be distinguished statistically from the normal frames. For INT data, pixels with values above a selected threshold, CR_thrshld, are flagged and the process is applied on all the frames.

$$\begin{aligned}\begin{array}{l}\texttt{if\,\,I}\_\texttt{bp}(\texttt{ix}, \texttt{jy}) > \texttt{CR}\_\texttt{thrshld,\,\,then}\\ \texttt{I}\_\texttt{cr}(\texttt{ix}, \texttt{jy}) = \texttt{INVALID}\_\texttt{PIX}\_\texttt{VALUE};\\ \texttt{else}\,\, \texttt{I}\_\texttt{cr}(\texttt{ix}, \texttt{jy}) = \texttt{I}\_\texttt{bp}(\texttt{ix}, \texttt{jy}); \end{array}\end{aligned}$$

for all pixels (ix, jy) of the frame.

For PC mode data, CR affected frames are first identified based on their event count being larger than a dynamically determined ‘event_count_threshold’ and then flagged. All events from these flagged frames are ignored for further processing. The value of ‘event_count_threshold’ is computed using two user selected parameters (p and q) as follows:

$$\begin{aligned}&\mathrm{event\_count\_threshold} \\&\quad =\mathrm{AVG + p*(AVG)^{0.5} + q/((AVG)^{0.5})} \end{aligned}$$

where AVG is an estimate for the average number of true photon events per frame determined from the entire dataset for that observation episode. The statistical scheme employed for finding AVG iteratively removes the CR affected frames from the sample as ‘outliers’. The second term in the equation addresses fluctuations in the case of very low values of event counts encountered for fields very dark in the UV. First, the CR affected frames are identified using their unique frame numbers and then all photon events from those CR affected frames are removed from the master list of events (and separately recorded in the log of CR affected frames). As a result, the total number of useful frames is reduced. The loss would depend on the orbit and distribution of matter in the spacecraft, and for UVIT it is $\sim$5 frames/s. Therefore, for optimal results this correction is only useful for dark fields where contribution of the cosmic rays to the photon counts is not much less than the contribution of the background.

1.5 Appendix A.5 uvtFlatFieldCorr

This Module, called by all the versions of the chains, applies a multiplicative correction for variations of response of the detector and transmission of the selected filter across the field. The correction factor arrays are available from the CAL_DB (FLAT_FIELDS_FILTER). The array appropriate for the band, mode and filter is selected to apply this correction. For INT mode this 2-D correction array, FF(ix, jy), is multiplied to every image frame, leaving flagged pixels (flagging process assigned an unique ‘invalid’ value for its easy identification) unaltered.

$$\begin{aligned}\begin{array}{l} \texttt{if\,\,I}\_\texttt{cr}(\texttt{ix},\texttt{jy}) =\texttt{INVALID}\_\texttt{PIX}\_\texttt{VALUE,\,\,then}\,\, \texttt{I}\_\texttt{ff}(\texttt{ix},\texttt{jy}) = \texttt{INVALID}\_\texttt{PIX}\_\texttt{VALUE};\texttt{else}\,\, \texttt{I}\_\texttt{ff}(\texttt{ix},\texttt{jy})=\texttt{I}\_\texttt{cr}(\texttt{ix},\texttt{jy})*\texttt{FF}(\texttt{ix},\texttt{jy}); \end{array}\end{aligned}$$

for all pixels (ix, jy) of the frame.

In the case of PC mode data, the value in ENP column for each event is replaced after multiplying it with the correction value from the array element corresponding to that event’s centroid coordinates.

$$\texttt{ENP}\_\texttt{ff}(\texttt{k}:\texttt{ix},\texttt{jy}) = \texttt{ENP}\_\texttt{uc}(\texttt{k}) * \texttt{FF}(\texttt{ix},\texttt{jy});$$

where integer part of the centroid coordinates of the event ‘k’ are (ix, jy); for all events in master list ‘k’ $=$ 0 to ‘l_max’.

1.6 Appendix A.6 uvtQEMCPCorr

This Module corrects for the thermal effects on the detectors response – through variation of Quantum Efficiency and gain of the Micro_Channel_Plate assembly. This Module is called by all the versions of the chains and it can be turned off. It uses Aux (LBT) data for reading the instantaneous temperature of the Detector Module (Camera Proximity Unit, CPU) and estimates corresponding multiplicative correction factor by interpolating the relevant table from CAL_DB (QE_TEMP). The in orbit thermal stability of the CPUs have been so good that so far this correction was never required.

1.7 Appendix A.7 uvtPixPadding

This Module is called by all versions of the processing chains. In order to accommodate movements of the UVIT field on sky due to spacecraft drift over the duration of Episode, it is necessary to enlarge the size of the 2-D arrays representing the sky images. For INT mode all image frames are expanded from $512\times 512$ to $600\times 600$ size, I_pp(ix, jy), by introducing a 44 pixel wide boundary on all four sides and assigning each of these added pixels with the value of ‘Bad Pixel’ flag. For the PC mode data, centroid coordinates (cx, cy) for all photon events are incremented by ($+$44, $+$44) along both axes:

$$\texttt{cx}\_\texttt{pp} = \texttt{cx} + \texttt{44}; \quad \&\quad \texttt{cy}\_\texttt{pp}= \texttt{cy} + \texttt{44};$$

for all events in master list ‘k’ $=$ 0 to ‘l_max’. It may be noted that the size of the padding chosen leads to a natural limitation of inclusion of the sky exposure data corresponding to spacecraft drifts at one go. However, a dataset with unusually large total drift can still be handled by sub-dividing it into multiple (non-overlapping) time ranges chosen intelligently based on the drift profiles along the two axes (X, Y). Its implementation involves selections of parameters for the Module uvtRefFrameCal described later on (see Appendix A.13). The pipeline will then need to be executed as many times as the number of sub-divided datasets.

1.8 Appendix A.8 uvtAccEveryTsec

This Module is called only by the Relative Aspect chain for Integration mode data (RA_INT). Functionality of this Module is to accumulate selected number, ‘N_acc’, of successive INT mode frames to enhance the signal for detection of faint stars with a good S/N ratio. A pixel by pixel averaging is carried out resulting in accumulated frames, Acc_frame-s. This Module does not handle PC mode (instead this functionality is embedded within the RA_PC chain directly).

1.9 Appendix A.9 uvtSubDivision

This Module is called by all versions of the processing chains, with the functionality of dividing the image pixels or their coordinates in to sub-pixels.

The target angular resolution for UV bands of UVIT was $<1.8^{\prime \prime }$, while the pixel scale for the final $512\times 512$ CMOS sensor is $\sim 3.3^{\prime \prime }$. The detector system and on board processing for PC mode ensures precision of determining event centroid coordinates (X, Y) to 1/32 of each pixel ($\sim 0.1^{\prime \prime }$). As a result, there is a need for sub-dividing various arrays. A subdivision by $8\times 8$ has been used to get more than 4 effective pixels within the required resolution (i.e., mapping $600\times 600$ to $4800\times 4800$). For PC mode, photon event centroids:

$$\texttt{cx}\_\texttt{sd} = \texttt{cx}\_\texttt{pp}*\texttt{8};\quad \& \quad \texttt{cy}\_\texttt{sd} =\texttt{cy}\_\texttt{pp}*\texttt{8};$$

for all events in master list ‘k’ $=$ 0 to ‘l_max’.

For INT mode processing, an user selectable factor, sub_div_factor, m ($=$ $2/4/8$) has been introduced for sub-division. This Module implements this sub-division where, original contents of the individual pixels are equally divided among the corresponding sub-divided pixels depending on value of sub_div_factor, m. The sub-pixels corresponding to flagged pixels are retained as flagged.

$$\begin{aligned}\begin{array}{l}\texttt{if\,\,I}\_\texttt{acc}(\texttt{ix}, \texttt{jy}) = \texttt{INVALID}\_\texttt{PIX}\_\texttt{VALUE,\,\,then}\,\\ \texttt{I}\_\texttt{sd}(\texttt{i}\_\texttt{m}\_\texttt{x}, \texttt{j}\_\texttt{m}\_\texttt{y}) = \texttt{INVALID}\_\texttt{PIX}\_\texttt{VALUE};\\ \texttt{else\,\,I}\_\texttt{sd}(\texttt{i}\_\texttt{m}\_\texttt{x},\texttt{j}\_\texttt{m}\_\texttt{y}) = \texttt{I}\_\texttt{acc}(\texttt{ix},\texttt{jy})/(\texttt{m}^{\wedge}\texttt{2});\end{array} \end{aligned}$$

over (m $\times$ m) sub-pixels; for all pixels (ix, jy) of the image and repeated for all the images.

1.10 Appendix A.10 uvtDetectStar

This Module is for identifying stars in an image and quantifying coordinates of their geometric centroids. It is called by all versions of the chains.

This Module processes individual sky images to identify candidate stars from local maxima in light distribution, confirm them to be stars from refined analyses of neighbourhood pixels and compute the centroid coordinates. Finally, an intensity ordered table of stars is generated. This Module supports both INT and PC mode data. The functionality of this Module is very basic and general purpose, accordingly it is called explicitly by the two chains extracting spacecraft aspect (RA_INT and RA_PC) and implicitly by the remaining two chains for their larger Modules: uvtRegAvg (Appendix A.19), uvtFullFrameAst (Appendix A.20), which are described later. Figure 15 presents a schematic flowchart for this Module. In brief, the implementation is as follows. The values of all valid pixels in the input sky image frame are used to determine the ‘mean (MN)’ and ‘standard deviation (SD)’ for the background, after removing ‘outliers’ (due to the presence of stars) by using an iterative scheme. An initial starting ‘threshold’ for detecting a star is defined as (MN $+$ p $*$ SD), where ‘p’ is user selectable and is initially kept at a conservatively high value ($\sim$10–50). At first all pixels with values above this threshold are considered to be associated with potential stars. If their number is lower than the user selected value for minimum number of stars, ‘min_stars’, then the value of ‘p’ is reduced gradually and the process repeated, till the requirement is met.

Figure 15

Flow chart for star detection (uvtDetectStar Module). The algorithm for detecting stars is based on identifying local peaks above a threshold which is sum of average background and the standard deviation multiplied by a factor, ‘p’. The value of ‘p’ is tweaked iteratively (initially starting with a high value) till the desired number of stars are identified. Centroid positions and intensity for the stars are tabulated.

Multiple neighbouring pixels at this stage could correspond to the same star. In the next step, a neighbourhood criterion using a square box foot print for the star image is used to identify the brightest pixel in each box to get a refined list of pixels (rp_ix, rp_jy). Size of the box is user selectable but the optimal size is found to be $15\times 15$. Whether the candidate mimics signatures of a genuine star is tested by an elaborate logic using the values of the neighbouring pixels, thus providing immunity against noise spikes. If the number in this refined list does not meet the ‘min_stars’ condition, then the value of ‘p’ is reduced further and both the steps described above are repeated till the condition is met (or ‘p’ reaches a value <5 and the process aborts with appropriate error log).

In the final step, intensity weighted centroid and background corrected total intensity in the box are computed. They are computed as

$$\begin{aligned}\begin{array}{l} \texttt{c}\_\texttt{l}\_\texttt{x}=[\texttt{sum}\{\texttt{sig}(\texttt{ix},\texttt{jy})*\texttt{ix}\}/\texttt{sum}\{\texttt{sig}(\texttt{ix},\texttt{jy})\}]\\ \texttt{c}\_\texttt{l}\_\texttt{y}=[\texttt{sum}\{\texttt{sig}(\texttt{ix},\texttt{jy})*\texttt{jy}\}/\texttt{sum}\{\texttt{sig}(\texttt{ix},\texttt{jy})\}]\\ \texttt{int}\_\texttt{l} = [\texttt{sum}\{\texttt{sig}(\texttt{ix},\texttt{jy})\}]\end{array} \end{aligned}$$

where sum is evaluated for ‘ix’ and ‘jy’ running through [rp_ix – (k+1)/2] to [rp_ix + (k+1)/2] and [rp_jy – (k+1)/2] to [rp_jy + (k+1)/2], respectively, where ‘k+2’ is size of the box and ‘Sig’ are background corrected values of Signal.

Finally, an intensity ordered (descending) table of intensities (int_l) and centroids (c_l_x, c_l_y) is generated.

1.11 Appendix A.11 uvtDetectDistCorr

This Module is called by both INT and PC versions of the Relative Aspect chain and the PC version of the Sky Imaging chain.

This Module applies corrections for the distortion inherent to the Detector Module (primarily due to the fibre optic taper). This distortion modifies the expected linear relation between location of electronic detection of a photon event and its original location of arrival on the photo-cathode. Hence, in order to determine the true position for each photon event, a correction needs to be applied. This correction has been quantified by extensive calibrations in the laboratory and confirmed in orbit using well studied astronomical targets. The corrections are identical for all the filters of the band and are available in the CAL_DB (DISTORTION/DETECTOR/band) as $512\times 512$ arrays. For PC mode, the correction is applied to centroids of every individual photon event and for INT mode to the centroids of each detected star tabulated by the uvtDetectStar Module (Appendix A.10). The correction values applicable to the nearest pixel coordinates for each centroid (c_x, c_y) are applied:

$$\begin{aligned}&\texttt{c}\_\texttt{x}\_\texttt{ddc} = \texttt{c}\_\texttt{x} -\\ &\quad \texttt{det}\_\texttt{dist}\_\texttt{corr}\_\texttt{x}\{\texttt{nint}(\texttt{c}\_\texttt{x}-\texttt{44}),\texttt{nint}(\texttt{c}\_\texttt{y}-\texttt{44})\};\\ &\texttt{c}\_\texttt{y}\_\texttt{ddc} = \texttt{c}\_\texttt{y} -\\ &\quad \texttt{det}\_\texttt{dist}\_\texttt{corr}\_\texttt{y}\{\texttt{nint}(\texttt{c}\_\texttt{x}-\texttt{44}),\texttt{nint}(\texttt{c}\_\texttt{y}-\texttt{44})\} \end{aligned}$$

where the arrays ‘det_dist_corr_x’ and ‘det_dist_corr_y’ are from CAL_DB (for appropriate band FUV/NUV/ VIS). An offset in indexing (by 44; see Appendix A.7) is needed since the arrays in CAL_DB are in $512\times 512$ system (and all coordinates subsequent to uvtPixPadding Module have them translated to $600\times 600$ system to accommodate spacecraft drift).

1.12 Appendix A.12 uvtOpticAssDistCorr

This Module is called by both INT and PC versions of the Relative Aspect chain and the PC version of the Sky Imaging chain.

This Module corrects for distortion due to the telescope optical assembly and is very similar to that discussed above (uvtDetectDistCorr in Appendix A.11). This distortion has been estimated from design of the telescope and is very small. The corrections to be applied are available in the CAL_DB (DISTORTION/OPTICS/band/filter). It may be noted that since this particular correction originates from design of the telescope optics alone, the CAL_DB arrays are identical for all filter-band combinations. For PC mode, the correction is applied to centroids of every individual photon event and for INT mode to the centroids of each detected star tabulated by the uvtDetectStar Module (Appendix A.10). The correction values applicable to the nearest pixel coordinates for each centroid (from the earlier Module, say, c_x_ddc, c_y_ddc) are applied:

$$\begin{aligned}&\texttt{c}\_\texttt{x}\_\texttt{oadc} = \texttt{c}\_\texttt{x}\_\texttt{ddc} -\\ & \texttt{optic}\_\texttt{assm}\_\texttt{dist}\_\texttt{corr}\_\texttt{x}\{\texttt{nint}(\texttt{c}\_\texttt{x}\_\texttt{ddc}-\texttt{44}),\\ & \texttt{nint}(\texttt{c}\_\texttt{y}\_\texttt{ddc}-\texttt{44})\};\\ & \texttt{c}\_\texttt{y}\_\texttt{oadc} = \texttt{c}\_\texttt{y}\_\texttt{ddc} -\\ & \texttt{optic}\_\texttt{assm}\_\texttt{dist}\_\texttt{corr}\_\texttt{y}\{\texttt{nint}(\texttt{c}\_\texttt{x}\_\texttt{ddc}-\texttt{44}),\\ \,& \texttt{nint}(\texttt{c}\_\texttt{y}\_\texttt{ddc}-\texttt{44})\} \end{aligned}$$

where the arrays ‘optic_assm_dist_corr_x’ and ‘optic_assm_dist_corr_y’ are from CAL_DB (for appropriate band FUV/NUV/VIS and filter F1/F2/$\ldots$) in $512\times 512$ system.

1.13 Appendix A.13 uvtRefFrameCal

This Module selects timing reference and step size for subsequent extraction of drift series. It is called by both INT and PC versions of the Relative Aspect chain. This Module is responsible for two functionalities: (1) generate the Reference Frame defined by an ensemble of centroids and intensities of the detected stars with respect to which all Relative Aspects for future instances of time will be computed and (2) to prepare tables, similar to the preceding ‘(1)’, of detected stars for all subsequent time samples. These are then used to find the drifts in a later processing Module (uvtComputeDrift in Appendix A.14). It begins with the tabulated lists of distortion corrected centroids of stars detected. Each table represents one time sample. The sequence of tables as a time series have been generated earlier from individual Acc_frame-s (by uvtDetectStar in Appendix A.10). There are two user selected parameters for this Module, namely, number of initial time samples to be ignored, N_skip, and the number of successive time samples to be combined as a block, N_avg, to effectively widen the time bin for computing the average positions of the stars. The resulting time series from the sequence of blocks will subsequently be used for computation of the drift. The very first block is used to generate the Reference Frame by associating stars using neighbourhood criteria on the centroids from N_avg number of individual Acc_frame-s. This is followed by recording the average values for centroid along X, centroid along Y, Intensity and Time. This process is repeated for the subsequent blocks. The selectable parameter N_skip provides a scheme to handle dataset with unusually large drift, by breaking down the full time range into multiple parts.

1.14 Appendix A.14 uvtComputeDrift

This Module is called by both INT and PC versions of the Relative Aspect chain. It supports both INT and PC mode input data and carries out the following tasks: (1) compute the drift which includes shifts and rotation of the field of selected UVIT band as a function of time in Detector coordinate system to generate Relative Aspect series (RAS); (2) time domain digital low pass filtering of the RAS using selectable parameters and (3) to translate the filtered RAS to the spacecraft coordinate system, namely, Roll, Yaw and Pitch using mechanical and optical mounting details of the bands on the spacecraft. Figure 16 presents the schematic flow of processing in this uvtComputeDrift Module.

The drift computation involves at first reading in all the tables of centroids and intensities for stars, from products of the earlier Module uvtRefFrameCal (Appendix A.13). In the next step, within a loop, two consecutive tables are considered which are referred to using subscripts ‘p’ and ‘c’ for ‘previous’ and ‘current’, respectively. The stars common to both the tables are identified and associated as pairs using their centroid coordinates and an user selected neighbourhood criterion representing maximum expected relative drift.

The generation of the RAS involves computation of best estimates the three variables ‘$\delta$x_shift’, ‘$\delta$y_shift’ and ‘$\delta \theta$’ in detector coordinate system, representing the best fit for transformation of centroids from ‘table_p’ to ‘table_c’. This best estimation is carried out using the centroids and a selectable algorithm (3 choices available). The most commonly used algorithm finds solution for least sum of squares of deviations, while giving equal weight to all the stars to avoid bias due to some bright stars in few locations. The algorithm is linearized by use of the approximation ‘$\sin \delta \theta = \delta \theta$ and $\cos \delta \theta =1$’ which is valid as the relative rotations within any Episode are $<0.1^{\circ }$.

Figure 16

Flow chart for extraction of drift (uvtComputeDrift Module). Using centroids of detected stars in successive frames (sampled every $\sim$1 s for VIS tracking) the incremental relative drift local in time is estimated first. The quantified drift includes two shifts along the detector axes and a possible rotation. Subsequently they are translated to cumulative total drift as a function of time, with respect to the reference frame. Time domain filtering is used for smoothen the three drift variables. Finally, they are transformed to spacecraft coordinate system (for use by any of the bands).

In order to limit the error propagation through addition of multiple incremental corrections over multiple time steps, the above scheme is repeated again between the star centroids in the Reference Frame (time_RF) and the drift corrected centroids for any time sample time_c obtained above. The additional corrections obtained now (say, $\delta \theta 2$, $\delta$x2_shift and $\delta$y2_shift) are used to compute the final drift values valid for time_c, as:

$$\begin{aligned}&\mathrm{{\theta }\_RF\_fin(time\_c) = {\theta }\_RF\_est(time\_c) + {\delta }{\theta }2}\\&\mathrm{x\_shift\_RF\_fin(time\_c)}\\&\quad \mathrm{= x\_shift\_RF\_est(time\_c) + }{\delta }\mathrm{x2\_shift}\\&\mathrm{y\_shift\_RF\_fin(time\_c)}\\&\quad \mathrm{= y\_shift\_RF\_est(time\_c) + }{\delta }\mathrm{y2\_shift} \end{aligned}$$

The complete process described above is repeated for the entire time range by advancing the time by steps of one in a loop. The resulting time series of $\theta$_RF_fin, x_shift_RF_fin and y_shift_RF_fin are the Relative Aspect series in detector coordinate system. Other choices of algorithm for extraction of drift parameters include: (a) dropping the assumption about ‘$\theta$’ to be small and (b) assigning intensity based weights to individual stars (weight proportional to intensity). In all the three algorithms, user has a choice to drop the parameter ‘$\theta$’ from drift extraction (i.e., consider only shifts and no rotation).

The time domain low pass filtering is carried out to minimize errors in the drift parameters. A sliding time window ($\Delta T$) selects the samples over ($t-\Delta T/2$) to ($t + \Delta T/2$) duration. They are then fitted with a polynomial of degree ‘n_poly’ using a least square method. The value for fitted polynomial at mid-point of the time window is assigned as the filtered result for time instance ‘t’. This process is repeated through the complete time series. While $\Delta T$ and n_poly are selectable by the user the most commonly used values for these have been 4 s and 1 s, respectively.

Finally, the drift parameters are transformed from the detector coordinate system to the spacecraft coordinate system using available transformation matrices. The convention followed for mapping them are: $\theta \ {\Rightarrow } \ \mathrm{Roll}, \ X \Rightarrow \mathrm{Pitch}$ and $Y \Rightarrow \mathrm{Yaw}$. Details of the two way transformations between the detector and spacecraft coordinate systems are available in Appendix C. The final Relative Aspect series, RAS, are made available in both the detector as well as spacecraft coordinate systems, which are required by both versions of the Sky Imaging chains (L2_PC and L2_INT). The standard bundle of the pipeline products includes the two RAS tables for every individual Episode, one each from RA_INT and RA_PC chains. Tables show the drifts at regular intervals of time.

1.15 Appendix A.15 uvtCentroidCorr

This Module was planned for the PC mode version of the Sky Imaging chain. It corrects for a subtle effect which introduces shift in the centroid of a photon event computed on board. It can arise if estimate of the background signal, which is equated to the lowest value of the four corner signals of $5\times 5$ window around centre of the event, used for on-board calculations of centroid differs significantly from the actual value due to a large noise. The mitigation scheme employed use of median filtered background from measured dark frames in orbit. During early operations in orbit, the noise in background was found to be insignificant. Accordingly, need for activating this Module has not arisen. This correction must precede the correction for centroid bias (uvtCentroidBias, Appendix A.16) described next.

1.16 Appendix A.16 uvtCentroidBias

This Module is called by both the Relative Aspect and Sky Imaging chains for the PC mode. It applies corrections for bias introduced on the centroids of photon events computed by on board signal processing. This systematic bias is inherent to the scheme adopted for obtaining the centroids from raw pixel values in the event foot print ($3\times 3$ or $5\times 5$), which slightly modifies the fractional part of the computed centroid value from the corresponding true value. Sometimes this effect is also referred to as Fixed Pattern Noise. The effects of this bias have been quantified from the extensive laboratory calibrations which have been translated into look up tables correlating the measured with the true values for the fractional part of the centroids along both the axes (X, Y). The CAL_DB provides these look up tables. This uvtCentroidBias Module recovers the true value from the measured one by applying the corresponding correction (for the appropriate band FUV/NUV/VIS and selected foot print $3\times 3/5\times 5$; e.g., BIAS_CORR/FUV$/$$3\times 3$) to all centroids in the master table of events.

1.17 Appendix A.17 uvtShiftRot Module and further updating of event table

This Module is called by both INT and PC versions of the Sky Imaging chain. The uvtShiftRot Module carries out the following two tasks for INT mode observations: (a) apply the drift to individual images – single frame or ‘N_acc’ frames clubbed together as Acc_frame in the Module uvtAccEveryTsec, to align these with the reference frame; (b) create Exposure arrays of the detector and aligning them to the reference frame by applying drift corrections. For the PC mode, only the first of these two functionalities is incorporated by applying the drift to the centroids of photons. In the PC mode, instead of Exposure arrays for individual exposure an integrated Exposure array is created by adding the Exposure arrays corresponding to individual images. This has been done as the number of individual images in an Episode could be about a million storing these individually would be too expensive in memory space.

1.17.1 Appendix A.17.1 Task-A

The drift parameters for every image are calculated by a linear interpolation of the entries in the table of drifts for the band, provided by A1.14 uvtComputeDrift, at the relevant time.

1.17.2 A.17.1a PC mode

For PC mode data, these interpolated drift parameters are used to apply identical corrections to centroids of all photon events of the specific frame.

$$\begin{aligned}&\mathrm{c\_x\_snr\,=\,x\_shift\,+\,(c\_x\,-\,x0)*cos}\theta \mathrm{\,-\,(c\_y -y0)\,*\,sin}\theta \mathrm{\,+\,x0 } \\&\mathrm{c\_y\_snr\,=\,y\_shift\,+\,(c\_x\,-\,x0)*sin}\theta \mathrm{\,-\,(c\_y -y0)\,*\,cos}\theta \mathrm{\,+\,y0} \end{aligned}$$

where x0 $=$ x_size/2 and y0 $=$ y_size/2; where the ‘snr’ subscript refers to the shift and rotation corrected centroids and the (c_x, c_y) are the most recently updated centroids from the master photon event list from the Module ‘uvtOpticAssDistCorr’ (Appendix A.12). These drift corrected centroid values (c_x_snr, c_y_snr) for each photon event replace the earlier entries for centroid values in the master table of events and remain available for subsequent processing blocks till the very end of execution of the chain. In the case the RAS does not cover the entire time range for the Episode, only the part with time overlap is retained and the remaining events are discarded since the drift corrections cannot be applied to them. This updated table is preserved in the sub-directory labeled ‘uvtShiftRot_6.3’ within the product bundle for the corresponding Episode, in a file named with the suffix ‘_snr’ (see Appendix I).

1.17.3 Further updating by a later processing (uvtFullFrameAst Module)

The photon event table finally updated by this uvtShiftRot Module (described above) gets further refined by a later processing Module carrying out astrometric corrections by matching detected stars with those in an existing standard catalogue (Module uvtFullFrameAst; Appendix A.20). A new event table is generated which adds fresh information regarding coordinates and time while retaining all the past details, located in the same sub-directory with the file name having the suffix ‘_radec’.

Five additional columns are appended, which are named ‘RA’, ‘DEC’, ‘MJD_L2’, ‘RA-aaaaa’ and ‘DEC-aaaaa’. The first pair contains the best estimate of the event coordinates in RA, Dec (ICRS; J2000) system relative to the map centre, which are the final values obtained from the pipeline run for the corresponding Episode including Astrometry. The last pair correspond to the RA, Dec obtained through transformation of event centroids from detector (X, Y) system to RA, Dec system using the aspect information from the spacecraft. In the event of failure of the astrometry block, both sets of coordinates would be identical.

The column named ‘TIME’ was planned to provide the primary timing information for each photon event. It is normally populated with SPS_MJD, the absolute time provided by L1 through calibration of the spacecraft’s on board clock, through selection of the ‘utcFlag’ to be ON. Alternately, if the UTC switch is selected to be OFF (when SPS_MJD showed erratic values occasionally), this ‘TIME’ column is populated with the UVIT’s Master clock, and the ‘MJD_L2’ column in the table would still provide estimated absolute time with about $\sim$1 s accuracy, using local calibration of the UVIT’s Master clock (carried out within the L2 pipeline, in DataIngest Module – Appendix A.1). This enables astronomical variability analysis at time scales shorter than typical duration of observation episodes which is $\sim$2000 s under all circumstances.

The standard bundle of the pipeline products disseminated by the ISSDC includes this final resulting table (with suffix ‘_radec’), as the event list table, corresponding to every dataset from individual Episodes.

1.17.4 A.17.1b INT mode

For INT mode data, individual image frames are first transformed by pixel sub-division from original size $4800\times 4800$ to $9600\times 9600$. Then contents of each of these finer sub-pixels are moved to a new location after application of the shift and rotation correction using relations similar to the above. Finally, a $2\times 2$ binning is carried out to bring the array dimensions back to original level. The process ensures local ‘conservation’ of signals and adequate handling of ‘Bad Pixels’. At the end, as many drift corrected image frame products are generated as were available at the input. They are all in Detector coordinate system (X, Y).

1.17.5 Appendix A.17.2 Task-B

For the second task of generating Exposure arrays, at first the relevant ‘template Exposure array’ is accessed from CAL_DB (EXPOSURE_TEMPLATE/band/mode/window size). Primarily, entries in this array are ‘1’ for all sub-pixels corresponding to the good pixels within the selected window and ‘0’ for the rest. For every image qualified for inclusion in making sky image, a corresponding Exposure array is generated by applying drift corrections (on the template array) appropriate for that time instance following the procedure described in Task-A (A.17.1b INT mode).

1.18 Appendix A.18 uvtFrameIntegration

This Module is called by both the Relative Aspect and Sky Imaging chains for PC mode data. It is responsible for gridding the photon centroids from multiple (N_combine) successive frames onto 2-D array/(s). For the RA_PC chain, a time series of images, Comb_frames, are generated which are used for identifying individual stars in a subsequent processing stage. For the L2_PC chain this Module is either used to combine all the frames for the Episode into a single image or construct multiple images by sub-dividing the full data into smaller time intervals corresponding to ‘pseudo-Episode’-s. The latter option enables application of corrections for small misalignments due to any temporal effects, in a subsequent processing step (Module uvtRegAvg; Appendix A.19).

The typical values of the time step ‘T’ corresponding to the selectable parameter, N_combine, are $\sim$1 s and $\sim$300 s for RA_PC and L2_PC chains, respectively. Time sequence of images, Comb_frames, are referred to as $4800\times 4800$ ‘Sig’ arrays. The last image would be made with < N_combine frames in the case the total number of frames is not integer multiple of N_combine. There is a provision to skip N_discard initial frames to discard the initial frames either due to any starting disturbance or during a rapid drift of the pointing. Corresponding sole/sequence of ‘Exposure’ array/s and ‘Uncertainty’ array/s are also generated in the L2_PC chain. The ‘Signal’ array/s are obtained by pixel by pixel division of the ‘Sig’ array/s by the corresponding ‘Exposure’ array/s and the statistical ‘Uncertainty’ array/s are generated using the ‘Signal’ and ‘Exposure’ array/s (based on square-root of estimated number of total EFFECTIVE photon events, corrected for the variation of responsivity on the detector, in each individual pixel and not the actual number of photon events). The ‘Exposure’ products from this Module are stored in the directory ‘uvtExposureFrames_6.3’ and the rest (Sig, Signal and Uncertainty) in sub-directories under the directory ‘uvtFrameIntegration_6.3’.

1.19 Appendix A.19 uvtRegAvg

This Module is called by both INT and PC versions of the Sky Imaging chain. It may be noted that in an earlier processing block (PC: uvtFrameIntegration Module; Appendix A.18; INT: uvtFindWtdMean for L2_INT (not described here), the dataset from a specific Episode has either from a specific Episode has either been treated as a whole (‘Episode’) or in smaller parts (‘pseudo-Episode’) as per users choice. Here, the Module uvtRegAvg, combines the multiple Signal (‘Sig’) and Exposure (‘Exposure’) arrays in the case ‘pseudo-Episode’ option was exercised in that earlier bloc. These multiple ‘Sig’ and ‘Exposure’ arrays are combined after determining relative Shifts and Rotation between them using an algorithm to align detected brightest point sources from successive ‘Sig’ arrays which were generated earlier on. The alignment operations are identical for each pair of ‘Sig’ and ‘Exposure’ arrays.

Additionally, for PC mode UV imaging, these combined ‘Sig’ and ‘Exposure’ arrays are used to generate the output ‘Signal’ and ‘Uncertainty’ arrays in true ‘count/s’ unit, using the same scheme as described earlier (uvtFrameIntegration Module; Appendix A.18). The same operation is carried out on the sole pair of ‘Sig’ and ‘Exposure’ arrays in the case the ‘Episode’ option was exercised in the preceding processing block to generate all three final products, namely, ‘Signal’, ‘Exposure’ and ‘Uncertainty’.

At the end of this uvtRegAvg process (irrespective of the choice between pseudo-Episode/Episode), a single set of sky image, exposure and uncertainty products are generated which utilize the entire dataset for the particular observation Episode (for a particular combination of band, filter and window size in one orbit). These final array products Signal (counts/s), Exposure (no of frames) and Uncertainty (counts/s) are of $4800\times 4800$ size and in detector coordinate (X, Y) system and stored in the ‘uvtRegAvg_6.3’ directory (see Appendix I).

Since the algorithms used in this uvtRegAvg Module for three different steps of processing are identical to those described earlier for other Modules, only their references are mentioned here. The scheme to find bright point sources in individual component images (from pseudo-Episodes) is similar to that used for the Module uvtDetectStar (Appendix A.10) and to determine relative offset (shifts and rotation) between them uses the scheme followed for the Module uvtComputeDrift (Appendix A.14). Finally, the scheme employed for aligning these individual images based on the locations of detected point sources, is the same as followed for the Module uvtShiftRot (Appendix A.17.1b; Task-A, for INT mode). It may be noted that the alignment of pseudo-Episodes involves transformation of image sub-pixels resulting in some loss of angular precision, unlike alignment among Episodes carried out in the Driver Scheme (Section 3.2.4) in which individual photon centroids are transformed.

1.20 Appendix A.20 uvtFullFrameAst

This Module is called by both INT and PC versions of the Sky Imaging chain as well as the Driver Scheme, which determines finer corrections to the sky coordinates of image products by correlating stars detected in the UV image with standard astronomical catalogues of stars.

The Module uvtFullFrameAst attempts to improve the accuracy of the aspect of the UV sky image products generated prior to invoking this Module. The input images have their sky coordinates determined based on information of spacecraft attitude (Roll_RA, Roll_DEC, Roll_ROT; ICRS J2000) provided in the Level 1 Aux data (‘*.att’ file in ‘mL1’ dataset). The typical accuracy of these coordinates is $\sim$45 arc-sec RMS and up to 3 arc-min peak-to-peak. Hence, there is a large scope for improvement by correlating stars detected in the image products with standard star catalogues to first determine the offsets and then applying corresponding corrections.

The algorithm followed is described here. First identify brightest point like sources detected from the UVIT image (NUV/FUV) and record their RA-Dec (ICRS) coordinates. They are then correlated with brightest entries in the USNO A2 optical star catalogue within a selected search radius ($\sim$3 arc-min). The USNO catalogue was reformatted by (a) retaining stars brighter than 16 magnitude and (b) embedding an indexing scheme to enable very fast search by the pipeline. ‘Good’ or ‘accidental’ matches of UVIT-USNO pairs are classified based on a sequence of statistical tests carried out iteratively. At first the brightest pairs are considered and progressively less bright pairs are also included for this analysis. If successful in identifying confirmed matches, a least square method is employed to extract the three parameters – shifts along RA and Dec and a rotation around the centre of the image (similar to the algorithm described for the Module uvtComputeDrift, Appendix A.14). The next step is to apply these corrections (shifts and rotation) to the three standard products, namely, UV image (‘Signal’), Exposure and Uncertainty arrays. The scheme for application of these astrometric corrections to the UV image differs between the case for individual Episode (when sub-division into pseudo-Episodes is not selected), from that for multi-Episode case. While the astrometric corrections are applied to centroids of individual photons in the former case, the shift and rotational transformations are applied on the sub-divided array elements for the latter.

1.21 In the Sky Imaging chain L2_PC

When this uvtFullFrameAst Module is called by the Sky Imaging chain L2_PC for an individual Episode, the corrections are applied to the centroids (RA, Dec) of each photon event and the master table of events is updated. Two additional columns are introduced in this master table, which hold the astrometry corrected coordinates (details given in Apppendix A.17 for Module uvtShiftRot, Task-A, PC mode). The final UV image is generated by populating these photons in Signal array of $4800\times 4800$ size. This process ensures minimal loss of precision (angular resolution) while applying astrometric corrections. Similar corrections for the Exposure and Uncertainty arrays are implemented by physical movements of pixel contents as described for Module uvtShiftRot (Appendix A.17, Task-A, INT mode; first pixel sub-division from $4800\times 4800$ to $9600\times 9600$; re-gridding individual sub-pixels by applying transformation; and finally $2\times 2$ binning).

1.22 In the Driver Scheme

When this uvtFullFrameAst Module is called by the Driver Scheme to apply astrometric corrections to the combined multi-Episode products, all three arrays namely, Signal, Exposure and Uncertainty are handled identically. In this case, Signal array also undergoes the same scheme of corrections for the two shifts and the rotation, as implemented for the Exposure and Uncertainty arrays in the case for individual Episodes described in the preceding para.

In addition to using the optical catalogue USNO A2, bright NUV and FUV source catalogues from GALEX mission are also used to correlate bright point like sources from UVIT images. The good matches of UVIT-GALEX pairs are also used to extract the offset corrections, but they are only recorded in a log file. The details of star match ‘success’/‘failure’ with Optical and UV catalogues are recorded as HISTORY/COMMENT entries on image HEADERs.

All the image products Signal (counts/s), Exposure (no of frames) and Uncertainty (counts/s) are of $4800\times 4800$ size and in RA-Dec (ICRS; J2000) coordinate system. They all are stored in the sub- directory named ‘uvtFullFrameAst_6.3’ either below a directory labeled by the specific Episode (see Appendix I) or a directory identifying the ‘group’ (Band/Filter/Window) for multi-Episode case (see Appendix J). In the case of failure of this astrometry block, the set of resulting products contain arrays identical to pre-astrometry images with adequate qualification in header keywords of the failure.

The standard bundle of pipeline products archived and disseminated by ISSDC includes the Signal image product from this Module as final UV sky image in Astronomical coordinates, corresponding to every Episode generated by the Sky Imaging chain (L2_PC), as well as combined multi-Episode images generated by the driver scheme for every ‘group’. The corresponding exposure and uncertainty products are also included.

Appendix B. Relative time shifts between the three bands

In spite of a common Master Clock used by all three bands of UVIT, the timing details of detector read out electronics lead to systematic shifts in the time stamps recorded on individual frames. These shifts depend on the imaging parameters configured. The relative time shifts most relevant for the pipeline are between the band used to obtain Relative Aspect series and the Imaging UV bands. This has been calibrated (in seconds unit) to be:

$$\texttt{t}\_\texttt{RAS}-\texttt{t}\_\texttt{img}=\texttt{t0}\_\texttt{RAS}-(-\texttt{T}\_\texttt{Img}\_\texttt{frame}/\texttt{2})$$

where ‘t0_RAS’ depends on the band generating RAS, its imaging mode and parameters; and ‘T_Img_frame’ is the frame integration time of the imaging Band (NUV/FUV); t0_RAS $=$ −0.427; for drift tracking by VIS band in INT mode; full $512\times 512$ field; t0_RAS $=$ −(T_RAS_frame/2); for drift tracking by NUV band in PC mode; any window size; where ‘T_RAS_frame’ is the frame integration time of the tracking band (NUV).

Appendix C. Transformations between the detector and spacecraft coordinate systems

The detector coordinate system for each band is defined by the X–Y axes of its electronic sensor (Star-250), and the axis normal to this plane corresponds to the optic axis of the telescope (either directly as for the FUV and VIS bands, or via one reflection with a plane surface; see Figure 1). The rotation angle about the optic axis is denoted by ‘$\theta$’ in the detector system. The Spacecraft coordinate system comprise of the axes Roll, Yaw and Pitch which is defined by the structure of the spacecraft. These two systems are interconnected through the details of mounting of sub-systems of UVIT on to the spacecraft’s mechanical interface. Nominally, the optic axis is aligned parallel to the Roll axis and the (X, Y) and (Yaw, Pitch) are connected through a single angle of rotation about an axis normal to all these four axes. Similarly, one rotation angle connects any pair of detectors for two bands. All these angles were initially measured during optical alignment tests in the laboratory. These were subsequently validated and refined by in orbit measurements during the Performance Verification phase.

The transformations between detector (X, Y, $\theta$) and spacecraft (Roll, Yaw and Pitch) are implemented as matrices for band, BND, as RPYTOXYTHETA_BND, {BND: FUV/NUV/VIS}, populated with numerical values, along with a corresponding plate scale. The reverse transformations, XYTHETATORPY_BND, are generated internally by matrix inversion.

• RPYTOXYTHETA_FUV: −0.00249798, 0.29636841, 0.29636841, 0.00249798

• RPYTOXYTHETA_NUV: 0.15423, 0.25153, −0.25153, 0.15423

• RPYTOXYTHETA_VIS: 0.1627, 0.2400, 0.2400, −0.1627

The above are consistent with the angles between $+$Y and the −Yaw (measured CCW) to be $+$0.483, $+$31.515 and $+34.134^{\circ }$ for the FUV, NUV and VIS bands, respectively. They are also consistent with the following directly observable relations which connect image coordinates of a star in the NUV and VIS detectors (accounting for slightly different plate scale for VIS; dX and dY are referenced with respect to the respective array centres):

• dX_NUV = 1.01652 * dX_VIS - 0.04649 * dY_VIS

• dY_NUV = - 0.04649 * dX_VIS - 1.01652 * dY_VIS

The corresponding relations connecting image coordinates of a star in the FUV and VIS detectors are:

• dX_FUV = -0.85093 * dX_VIS + 0.56645 * dY_VIS

• dY_FUV = 0.56645 * dX_VIS - 0.85093 * dY_VIS

Appendix D. Directory structure of merged Level 1 dataset (mL1)

Appendix E. Directory structure of UVIT calibration database (CAL_DB)

Appendix F. Directory structure for DataIngest output (Integration mode; INT)

Appendix G. Directory structure for DataIngest output (photon counting mode; PC)

Appendix H. Directory structure for the outputs from the Relative Aspect Chain for Integration Mode (RA_INT)

Appendix I. Directory structure for the outputs from the Level 2 Sky Imaging Chain for Photon Counting Mode (L2_PC)

Appendix J. Top level directories for products from Driver Scheme (with some additional details for combined multi-Episode products)

Appendix K. All user selectable parameters

List of all user selectable switch settings and parameters for the pipeline (through Parameter Interface Library, PIL), are presented in a tabular form (see supplementary material, Table 7 is available on the Journal of Astrophysics and Astronomy website https://www.ias.ac.in/Journals/Journal_of_Astrophysics_and_Astronomy/). The sequence of parameters are grouped by: DRIVER_SCHEME and RA_INT, RA_PC and L2_PC chains. Parameter names in bold font indicate them to be tweaked frequently by users.