Scribd Logo
Upload

Welcome to Scribd!

  • 96 views

    09 Incorta Administration

    Uploaded by

    Jaghan
    This document provides an overview of Incorta Analytics administration including prerequisites, architecture, data load processes, application monitoring, memory management, backups and restores, and the Incorta user interface. Key topics covered include installation requirements, important configuration files and commands, full versus incremental data loads, monitoring tools, best practices for memory usage, tenant management tools, and administrative functions available through the Incorta UI.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    09 Incorta Administration

    Uploaded by

    Jaghan
    96 views59 pages
    This document provides an overview of Incorta Analytics administration including prerequisites, architecture, data load processes, application monitoring, memory management, backups and restores, and the Incorta user interface. Key topics covered include installation requirements, important configuration files and commands, full versus incremental data loads, monitoring tools, best practices for memory usage, tenant management tools, and administrative functions available through the Incorta UI.

    Original Title

    09_Incorta_Administration

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document provides an overview of Incorta Analytics administration including prerequisites, architecture, data load processes, application monitoring, memory management, backups and restores, and the Incorta user interface. Key topics covered include installation requirements, important configuration files and commands, full versus incremental data loads, monitoring tools, best practices for memory usage, tenant management tools, and administrative functions available through the Incorta UI.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    96 views59 pages

    09 Incorta Administration

    Uploaded by

    Jaghan
    This document provides an overview of Incorta Analytics administration including prerequisites, architecture, data load processes, application monitoring, memory management, backups and restores, and the Incorta user interface. Key topics covered include installation requirements, important configuration files and commands, full versus incremental data loads, monitoring tools, best practices for memory usage, tenant management tools, and administrative functions available through the Incorta UI.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 59

    Incorta Analytics

    Administration
    Agenda

    • Prerequisites
    • Architecture & Installation
    • Data Load
    • Application Monitoring
    • Memory Management
    • Backup / Restore
    • Incorta User Interface
    Prerequisites
    • Python version V2.6 or later : python -V
    • Linux 64-bit. Mac OS 10.9 and 10.10. Windows 64-bit.
    • At least 16 GB RAM
    • Data on disk stored in compressed format
    • RDBMS for metadata Derby, MySQL, Oracle
    • Java SE Development Kit 8 or higher
    Ports Requirement
    Important Commands and Files

    Manage Memory Allocated to Incorta

    Edit Incorta_Installation/start.sh
    export JAVA_OPTS="-Xms16384m -Xmx16384m -Dfile.encoding=UTF-8"

    Start/Stop Incorta
    ./start.sh ./stop.sh

    Modify Ports
    Edit file Incorta_Installation/server/server.xml
    Modify: <Connector port="8080" to change the port

    Log File
    Incorta_Installation/server/logs/catalina.out
    Incorta_Installation/server/logs/incorta/<tenantname>/<incorta.YYYY-MM-DD.log >

    Tenant Directory
    Incorta_Installation/tenants/<tenant name>

    TMT Directory
    Incorta_Installation/tmt
    Data Loads
    Data Load
    Full vs Incremental
    • Full Load is done initially to extract full data from source system
    • Incremental to extract subsequence updates/inserts
    • Scheduling Incremental Load
    • Incremental Refresh frequency vs Performance
    Data Load
    Data Load
    Application Monitoring
    Application Monitoring
    Nagios
    • Open Source Tool
    • Comes with standard functionality to monitory Server and databases
    • We can write shell scripts to monitor Tomcat Server and Memory Usage
    • Can send alerts via email and SMS
    Memory Management
    Memory Management
    Memory Usage using test.jsp
    http://ebsdev.incorta.com:8081/incorta/test.jsp
    Memory Management
    Monitoring Server Status / Memory from UI

    http://ebsdev.incorta.com:8081/manager/html
    Memory Management

    Best Practices
    • Select required Columns instead of doing Select *
    • Use appropriate data types. For example Integer vs Long
    • Test.jsp for diagnosing memory usage
    • Hybrid Mode if data exceeds 75% of allocated memory
    • Load tables from staging when load fails due to out of memory issue
    Spark

    Why use Spark?


    MV Spark Parameter Configuration
    Monitoring Spark through WebUI
    Sparking OS Processes
    Spark Job Logs
    Spark

    Spark Integration complements Incorta by allowing Advanced


    Transformations:


    ◦ Complex functions (e.g. Window Functions …)


    ◦ Procedural Transformation Logic (using Python)
    ◦ Predictive Analytics, Machine Learning, ...
    Spark
    MV Spark Params
    Spark
    WebUI
    Spark
    Linux Processes

    Job Logs
    Backup / Restore
    Backup / Restore

    • Tenant Management
    • CLI Commands
    Tenant Management
    Tenant Management
    List Tenants

    ./tmt.sh -l

    List Tenant Properties

    ./tmt.sh -lc <tenant name>

    Update Tenant Property

    ./tmt.sh --update-property <tenant name> <proprerty> <value>


    ./tmt.sh --update-property <tenant name> load-mode demand

    Disable Tenant

    ./tmt.sh -d demo01

    Remove Tenant

    ./tmt.sh -r demo01
    Tenant Management
    Tenant Export: Exports tenant metadata into a zip file.
    • Creates a properties file for the tenant that can be used for quick edits and import.
    • Optionally export the data files as well

    Command:
    ./tmt.sh [-e,--export] <tenant_name> <export_file > [-cf]

    Parameters:
    tenant_name: Name of the tenant to export
    export_file : path to tenant export file [ defaults to tenant_name.zip ]

    Optional Parameters:
    [ -cf export data files ]

    Examples:
    Export tenant
    ./tmt.sh export tenant_name package_name.zip

    Export tenant with data files


    ./tmt.sh export tenant_name package_name.zip -cf
    Tenant Management
    Tenant Import: Imports Tenant from zip file

    Command
    ./tmt.sh [-i ,--import] <Tenant_Zip_File>

    Parameters
    Tenant_Zip_File: File to be imported

    Optional Parameters
    [ -on <new_name> override name ] import tenant with a new name
    [ -op <new_path> override path] import tenant with new location for tenant data

    Examples:
    Import tenant:
    ./tmt.sh --import Tenant_Zip_File
    Import tenant with new name:
    ./tmt.sh --import <Tenant_zip_file>. -on <New Tenant Name>
    Import tenant with new name and new path:
    ./tmt.sh --import <Tenant_zip_file>. -on <New Tenant Name> -op <New Path>
    Exercises
    List Tenants
    ./tmt.sh -l

    Export Tenant
    ./tmt.sh --export demo demo<n>.zip -cf

    Import Tenant
    ./tmt.sh --import demo<n>.zip -on demo<n> -op <Incorta_Home>/tenants

    List Tenant Properties


    ./tmt.sh -lc demo

    Update Tenant Property


    ./tmt.sh --update-property demo load-mode demand

    Remove Tenant
    ./tmt.sh -r demo<n>
    Command Line Interface
    (Export/Import various objects)
    CLI Commands
    • Login / Logout

    • Export / Import Data Sources

    • Export / Import Schemas

    • Export / Import Session Variables

    • Export / Import Dashboards

    • Load Schemas
    CLI Interface

    • Incorta provide CLI commands to export and import various objects.


    • Script is located under <Incorta_Installation>/bin directory

    Create login session using the following command:


    session=$(<Incorta_Installation>/bin/incorta.py login <Incorta URL> Tenant UserName Password)

    Verify that the session was created successfully


    (echo $session. It will display a string containing: {"verify":true})

    Logout from Incorta


    $<Incorta_Installation>/bin/incorta.py logout $session
    CLI Interface

    Export Data Source

    <Incorta_Installation>/bin/incorta.py export_datasources $session "<export_file.zip>" <DataSource1> <DataSource2>


    CLI Interface

    Import Data Source

    /home/incorta/Inc25/bin/incorta.py import_datasources $session "export_file.zip" <true>


    CLI Interface

    Import Data Source

    /home/incorta/Inc25/bin/incorta.py import_datasources $session "export_file.zip" true


    CLI Interface

    Export Schemas

    /home/incorta/Inc25/bin/incorta.py export_schemas $session "export_schemas.zip" HR SALES


    CLI Interface
    Import Schemas

    /home/incorta/Inc25/bin/incorta.py import_schemas $session "export_schemas.zip" true


    CLI Interface
    Export Session Variables

    /home/incorta/Inc25/bin/incorta.py export_session_variables $session “session_variables.zip" Session_Variable

    Exported to: session_variables.zip

    Import Session Variables

    /home/incorta/Inc25/bin/incorta.py import_session_variables $session “session_variables.zip"

    Exception: Object exists already


    CLI Interface

    Export Dashboard

    /home/incorta/Inc25/bin/incorta.py export_dashboards $session "export_dashboards.zip" "/dashboard path"


    CLI Interface

    Import Dashboard

    /home/incorta/Inc25/bin/incorta.py import_dashboards $session "export_dashboards.zip" true


    CLI Interface

    Load Schema
    /home/incorta/Inc25/bin/incorta.py load_schema $session <Schema Name>
    Exercises
    CLI Exercises

    Login and Logout from Incorta using CLI

    Export Data Source

    Import Data Source


    CLI Exercises

    Export Schemas

    /home/incorta/Inc25/bin/incorta.py export_schemas $session "export_schemas.zip" HR SALES

    Import Schemas

    /home/incorta/Inc25/bin/incorta.py import_schemas $session "export_schemas.zip" true

    Export Dashboard

    /home/incorta/Inc25/bin/incorta.py export_dashboards $session "export_dashboards.zip" "/dashboard


    path"

    Import Dashboard
    /home/incorta/Inc25/bin/incorta.py import_dashboards $session "export_dashboards.zip" true
    CLI Exercises
    Export Session Variables

    Import Session Variables


    Incorta User Interface
    Administration from Incorta UI

    Managing Users
    Administration from Incorta UI

    Managing Groups
    Administration from Incorta UI

    Roles
    Administration from Incorta UI

    Sharing Objects
    Administration from Incorta UI

    Export Schema
    Administration from Incorta UI

    Import Schema
    Administration from Incorta UI

    Export Dashboard
    Administration from Incorta UI

    Import Dashboard
    Configuration from Incorta UI

    http://ebsdev.incorta.com:8081/incorta/admin/
    Debugging
    Debugging Issues Using Log Files

    Server Log

    The main server log file catalina.out is located under <Incorta_Installation_Directory>/server/logs/ directory.

    Sample Errors

    ERROR finding Column: EBS_FA.FA_DEPRN_DETAIL.COST_ACCOUNT_CCID


    Missing parents for EBS_FA.FA_BOOKS-EBS_FA.FA_BOOK_CONTROLS: 157 of 11444
    Error: Column types are not compatible in join: EBS_GL.GL_JE_HEADERS-
    EBS_CAL_COMMON.GL_DATE_PERIOD_MAP
    ERROR: Unknown field name in formula: ebs_ap.ap_invoices_all.invoice_date
    ERROR: [22:13:14] [admin | POST/service/schema/loadData | ETL | EBS_GL | loading | EBS_GL.GL_JE_LINES]
    INC_090001003: Not enough memory to load schema EBS_GL.GL_JE_LINES
    [com.incorta.loader.LoaderJob.setError]
    Debugging Issues Using Log Files

    Tenant Logs

    Tenant Logs are located under <Incorta_Installation_Directory>/server/logs/incorta/<tenantname> directory.


    Each day a new log file is generated with date stamp.
    This log is very useful for debugging data extraction and load errors.

    Sample Errors

    WARN: [08:54:20] [admin | POST/service/schema/loadData | ETL | EBS_AP | extracting | EBS_AP.AP_HOLDS_ALL]


    Error in row #: 4 -> [HELD_BY : LONG : SFX_ADMIN : For input string: "SFX_ADMIN"]
    [com.incorta.loader.LoaderJob.extractTable]

    ERROR: [08:37:01] [admin | GET/service/folder/getChildren] INC_002001103:You are not authorized to VIEW the
    FOLDER of ID [527]. [com.incorta.server.catalog.CatalogManagerimpl.getFolderChildren]

    You might also like