# Adaptivestone Framework - Complete Documentation > This file is auto-generated from the Docusaurus documentation. > Generated on: 2026-01-17T21:22:27.788Z > Total documents: 17 --- ## Table of Contents 1. 01-intro 2. 02-configs 3. 03-files-inheritance 4. 04-base 5. 05-models 6. 05-modelsOld 7. 06-Controllers / 01-intro 8. 06-Controllers / 02-routes 9. 06-Controllers / 03-middleware 10. 07-logging 11. 08-i18n 12. 09-testsing 13. 10-cli 14. 11-cache 15. 12-email 16. 13-deploy 17. 14-helpers --- # Document 1: 01-intro # Intro Welcome to the AdaptiveStone framework documentation. We hope that the documentation is clean and short. Please feel free to edit it via a Git merge request and contact us. ## History You must be wondering - why another framework, as we currently have a lot of them? But each of them is different. The Adaptive Stone framework was born around 2016 and has been growing since. There were a few requirements that no one existing framework could provide us for SAAS at that time - working with multiple databases, scaling out of the box, and a modern approach. That is why the Adaptive Stone framework was born. ## Features - Automatically initialized controllers (no config needed) - Mongoose models - Winston logger (Sentry logger as a part of Winston logger) - Node cluster enabled by default (possibility to use with no cluster (dev mode)) - Docker development environment (and production too) - Integrated Biome (moder alternative to ESLint and Prettier) for code style - Cache system - Ability to overwrite any controller, model, and config that came with the framework - Multi-language support out of the box - ESM and CommonJS compatibility - TypeScript support (you are able to write everything in JavaScript, but in JS you will have types as a bonus) ## Folder Structure ```js framework/ ├─ commands/ // Contains the command folder (CLI) ├─ config/ // Contains config files ├─ controllers/ // Contains controller files │ ├─ {controller_group_folder}/ // Can contain a folder (will be added to the route) ├─ locales/ // i18n folder with translations │ ├─ {locale_1}/ // Locale name (en, fr, etc.) │ │ ├─ translation.json // Translation JSON file │ ├─ {locale_2}/ │ │ ├─ translation.json ├─ migrations/ // Folder where migration files are stored ├─ models/ // Contains model files ├─ modules/ // Main folder with abstract stuff ├─ public/ // Public stuff (served statically) ├─ services/ // Some services (email, http, etc.) ├─ tests/ // Folder contains basic tests ├─ views/ // The framework is able to respond with a view (not only API). View files are stored here. ├─ Cli.ts // Main CLI class ├─ cliCommands.ts // CLI implementation ├─ cluster.ts // Entry point for production for the cluster module ├─ folderConfig.ts // Folder configuration ├─ server.ts // Main entry point to the framework ``` ## Framework Structure ![Framework](/img/AdaptiveStroneFramework.jpg) ## Getting Started Get started by **creating a new project**. The simplest way to create a new project is to clone the **template** and customize it: ```shell git clone git@gitlab.com:adaptivestone/example-project.git adaptivestone-example-rename-me ``` ## Start Your Project You should have **[Docker](https://www.docker.com/products/docker-desktop)** and **[Docker Compose](https://docs.docker.com/compose/install/)** installed. Run the development server: ```shell cd adaptivestone-example-rename-me docker compose up ``` Your site starts at `http://localhost:3300`. Open `src/controllers/Person.ts` and edit some lines: the site **reloads automatically** and applies your changes. ## TypeScript Support The framework itself is written in TypeScript (erasable syntax), but the project is flexible to use either TypeScript or JavaScript. You are able to use the modern Node.js runtime to run TypeScript files without any compilation. You are also able to mix TypeScript and JavaScript files in one project. Just remember that types will only work if the imported file is a TypeScript file and it is imported into a TypeScript file. --- # Document 2: 02-configs # Configs Every modern application runs on multiple environments and should have the ability to configure itself without changing the code. ## Config Files Config files are located in ‘src/config/\{filename\}.(js|ts)’. Each config file is mostly unique, and you should refer to the module documentation (or the framework config section). :::note Config files are part of the [framework inheritance process](03-files-inheritance.md). ::: ## Environment Variables Out of the box, the framework supports environment variables with [process.loadEnvFile](https://nodejs.org/api/process.html#processloadenvfilepath). This is part of Node.js since v20.12.0. The framework will grab and parse the .env file in the root of the project directory to fill environment variables. The sample project ships with a basic .env.example file. :::tip Do not use environment variables in the code. Use them only inside config files. This will allow you to track variables and split configs and the codebase. ::: :::tip Do not push the .env file to the version control system, as it can contain passwords and keys. The file should be unique per environment. ::: :::tip Try not to overload the .env file with configuration and keep only private data in it. To manage non-private data, check the [NODE_ENV](#node_env) section. ::: ## NODE_ENV Depending on the NODE_ENV environment variable, the framework will load additional config files. Data from these files will be merged into the main config with an overwrite. This is useful when you want to have something that depends on the environment but do not want to overload the .env file. Also, it's a good practice to keep the basic config in files rather than in the .env file. **Example:** ```js title="/src/config/sample.ts" export default { variable1: 1, variable2: 2, }; ``` ```js title="/src/config/sample.production.ts" export default { variable2: 3, variable3: 4, }; ``` On a NON-production environment: ```js const sampleConfig = this.app.getConfig("sample"); // variable1: 1, // variable2: 2 ``` On a PRODUCTION environment: ```js const sampleConfig = this.app.getConfig("sample"); // variable1: 1, // variable2: 3, <-- DIFFERENT // variable3: 4 <-- NEW ``` You can use the same approach for different environments (dev, stage, testing, etc.). ## API ```ts getConfig(configName: string): {}; updateConfig(configName: string, config: {}): {}; ``` Returns the config based on the config name. It also caches it in memory. ```js const sampleConfig = this.app.getConfig("sample"); ``` Updates the config based on the config name. Returns the updated config. ```js const sampleConfig = this.app.updateConfig("sample", { variableToUpdate: 3, anotherVariable: 4, }); ``` --- # Document 3: 03-files-inheritance # File Inheritance The framework provides a flexible way to overwrite some functionalities from the core. You can create a file with the same name in the same folder, and the framework will use this file instead of the built-in one. Let’s go with an example. We have a User model built directly with the framework with some user-related stuff. We want to provide a fully different implementation of this model. To do that, we will create a user model file at the project level in the model folder. ```js project/ ├─ node_modules/ │ ├─ @adaptivestone/ │ │ ├─ framework/ │ │ │ ├─ models/ │ │ │ │ ├─ User.ts // Built-in model. Mark it as "User_original" ├─ src/ │ ├─ models/ // Contains model files │ │ ├─ User.ts // File that will be used. Mark it as "User_project" ``` ```js const User = this.app.getModel("User"); // Will return the "User_project" model ``` This also happens at **all** levels of the code. If some code inside the framework asks for the “User” model, it will get the “User_project” model. The same approach works for models, controllers, and configs. ## How To ### Extend a module with new functionality instead of completely overwriting it That's easy. Just require the original file and extend it. Please note: TypeScript types are fully optional. You are able to use plain JavaScript too. ```ts import OriginalUserModel from "@adaptivestone/framework/models/User.ts"; class User extends OriginalUserModel { static get modelStatics() { type UserModelLite = GetModelTypeLiteFromSchema< typeof User.modelSchema, ExtractProperty >; return { ...OriginalUserModel.modelStatics, // Grab the original model methods getPublic: function getPublic(this: InstanceType) { return { userName: this.name, }; }, }; } } ``` ### Disable functions completely To disable something (like a default controller), the best way is to overwrite it with an empty implementation. ```js import AbstractController from "@adaptivestone/framework/modules/AbstractController.js"; class Auth extends AbstractController {} export default Auth; ``` --- # Document 4: 04-base # Base Class Each class extends the Base class to have some basic functions, mostly related to logging and the inheritance process. The Base class implements on-demand logging instance loading. That means the logger will be united only when you first request it. This is an important part of loading speed optimization. ## API ```js class A extends Base { async someFunction() { // Access to the logger instance (please follow the logger documentation for more details). this.logger; // Get files with inheritance. const files = await this.getFilesPathWithInheritance( `${__dirname}/../../migrations`, this.app.foldersConfig.migrations ); } /** * Returns the logger group. Just to have all logs grouped logically. */ static get loggerGroup() { return "Base_please_overwrite_"; } /** * In case of logging, sometimes we might need to replace the name. */ getConstructorName() { return this.constructor.name; } } ``` ```js getFilesPathWithInheritance(internalFolder, externalFolder); ``` Will scan two folders and provide a path with inheritance. If the same file is present in both paths, the priority will be given to the external file. --- # Document 5: 05-models # Models The framework is based on the [Mongoose](https://mongoosejs.com/) library and provides direct access to it. The model handles database connections. :::note Model files are part of the [framework inheritance process](03-files-inheritance.md). ::: The model uses a class with static methods and properties, providing auto-typing for each model. TypeScript helpers are also available to extract types from the model class. ## Lifecycle The framework can do the following: 1. Load model files 2. Initialize model files There are internal options for this (see the Commands section), which are mostly used in commands where you can skip model initialization or load the model without initializing it. This is useful in a few use cases, primarily for type generation. Under normal conditions, the framework scans the `model` folder and loads all models using the inheritance process. This is primarily to avoid model 'circular dependencies.' ## Base Model The base model is the core of your models. It handles their structure and initialization. We have provided a TypeScript example, but you can ignore the types if you only want to use JavaScript. Using types is fully optional. ```ts import { BaseModel } from "@adaptivestone/framework/modules/BaseModel.js"; // in case you need to access appInstance - appInstance.getConfig('s3'); import { appInstance } from "@adaptivestone/framework/helpers/appInstance.js"; // These are TypeScript helpers. import type { GetModelTypeFromClass, // `GetModelTypeFromClass` returns model types from the class. GetModelTypeLiteFromSchema, // Same as above, but only uses the schema to avoid circular linking. } from "@adaptivestone/framework/modules/BaseModel.js"; import type { Schema } from "mongoose"; // Type helper for static and instance methods. type SomeModelLite = GetModelTypeLiteFromSchema; class SomeModel extends BaseModel { static initHooks(schema: Schema): void { // A place to initialize plugins, indexes, and so on. // This happens after the class is loaded into Mongoose but before Mongoose initializes it. // schema.plugin(PLUGIN_NAME); schema.index({ name: "text" }); // or indexes. // For hooks, there are two types of `this`: model and queries. // https://mongoosejs.com/docs/middleware.html#types-of-middleware schema.pre( 'save', async function (this: InstanceType) { ... } ); schema.pre('findOneAndDelete', async function () { const docToDelete = await this.model.findOne( // Helps to return the correct model. this.getQuery(), ); }); } // The Mongoose schema goes here. // This is a complete Mongoose schema. // Please refer to the Mongoose documentation: https://mongoosejs.com/docs/guide.html static get modelSchema() { return { someString: { type: String, required: true }, firstName: String, lastName: String, email: String, orders: { type: mongoose.Schema.Types.ObjectId, // This is the correct type for an ObjectID reference. Only this type generates valid types. ref: "Order", // You don't need to worry about initialized model schemas; the framework will load and initialize all models for you. }, } as const; // This helps generate better types (TypeScript only). } // The Mongoose schema options go here. // Please refer to the Mongoose documentation: https://mongoosejs.com/docs/guide.html#options static get schemaOptions() { return { read: "primary", } as const; // This helps to generate better types. (TypeScript only) } /** * Object with static methods. * this.app.getModel('SomeModel').findByEmail('email'); * this.app.getModel('SomeModel').getInfoStatic(); * */ static get modelStatics() { type OrderModelType = GetModelTypeFromClass; // To help with the `populate` method. return { findByEmail: async function findByEmail( this: SomeModelLite, // A type helper to map to the correct `this` context. email: string ) { const instance = await this.find({ email }); return instance; }, getInfoStatic: async function getInfoStatic( model: InstanceType // A TypeScript type helper. ) { await model.populate("orders"); return { _id: model.id, email: model.email, }; }, getInfoStaticWithOrders: async function getInfoStatic( // Intercepts model types to ensure that the `orders` type is correct (without interception, it will be just an `ObjectID`). model: InstanceType & { orders: InstanceType[]; } ) { await model.populate("orders"); return { _id: model.id, email: model.email, orders: model.orders, }; }, }; } /** * We should also have instance methods for the model to interact with. * const SomeModel = appInstance.getModel('SomeModel'); * const someModel = await SomeModel.findOne({email:"cfff"}); * const data = await someModel.getInfo(); // call instance method */ static get modelInstanceMethods() { type ShippingInstanceType = InstanceType; return { getInfo: async function getInfo(this: SomeModelLite) { return { _id: this._id, email: this.email, }; }, // anotherMethod, }; } /** * We should also have virtual methods for the model to interact with. * const SomeModel = appInstance.getModel('SomeModel'); * const someModel = await SomeModel.findOne({email:"cfff"}); * const fullName = await someModel.fullName // virtual field * someModel.fullName = 'Jean-Luc Picard'; */ static get modelVirtuals() { return { fullName: { // virtual field options: { type: Object, // schema }, get(this: InstanceType) { // Getter return `${this.firtsName} ${this.lastName}`; }, async set(this: InstanceType, v: string) { // Setter const firstName = v.substring(0, v.indexOf(" ")); const lastName = v.substring(v.indexOf(" ") + 1); this.set({ firstName, lastName }); }, }, }; // make sure that you not put it as a const } } export default SomeModel; // It's good practice to return the type from the model. export type TSomeModel = GetModelTypeFromClass; ``` :::warning Models are instantiated once per process, and their instances are then cached. Do not expect constructor or `init` hook calls on every model load. ::: :::tip Please do not use the plural form for model names. **Bad** - Coin**s** **Good** - Coin ::: ## API ```ts getModel(modelName: string): MongooseModel; ``` Example: ```js const UserModel = this.app.getModel("User"); const userInstance = await UserModel.findOne({ email: "user@email.com" }); ``` ## Configuration The main configuration variable is the `MONGO_DSN` environment variable, which the model uses to connect to the database. ## Built-in Models The framework comes with a few built-in models. ### User It is part of the authorization system and handles user storage, password hashing, and provides basic functions for token generation and user retrieval. If you want to create your own user implementation, you should override or disable this one. The authentication controller depends on this model. #### API ```js const UserModel = this.app.getModel("User"); const user = await UserModel.getUserByEmailAndPassword("email", "password"); const userToken = await user.generateToken(); // Generates and stores a token in the database const userPublic = await user.getPublic(); const hashedPassword = await UserModel.hashPassword("password"); const sameUser = await UserModel.getUserByToken(userToken); const sameUserAgain = await UserModel.getUserByEmail(user.email); const recoveryToken = await UserModel.generateUserPasswordRecoveryToken(user); const sameUserAgain2 = await UserModel.getUserByPasswordRecoveryToken( recoveryToken ); const isSuccess = await user.sendPasswordRecoveryEmail(i18n); const verificationToken = await UserModel.generateUserVerificationToken(user); const sameUserAgain3 = await UserModel.getUserByVerificationToken( verificationToken ); await UserModel.removeVerificationToken(verificationToken); const isSuccess2 = await user.sendVerificationEmail(i18n); ``` ### Migration The migration model is a helper for the migration subsystem. It stores the names of migrated files to ensure that each migration is only executed once. Please refer to the `CLI/migrations` section for more details. You should probably not use this model directly. ### Sequence The Sequence model allows you to generate sequences by name. This is a cross-server-safe method for generating sequences in a distributed environment. ```javascript const SequenceModel = this.app.getModel("Sequence"); // Will be 1. const someTypeSequence = await SequenceModel.getSequence("someType"); // Will be 2. const someTypeSequence2 = await SequenceModel.getSequence("someType"); // Will be 1, as the type is different. const someAnotherTypeSequence = await SequenceModel.getSequence( "someAnotherType" ); ``` ### Lock The Lock model provides the ability to lock resources in a distributed environment. This can be used for external requests, system actions, etc. Imagine you have a high volume of traffic requesting data from an external system. You also have a cache for this data, but you must initially query the internal API to retrieve it. To prevent overwhelming the API, you want to ensure that you only request the data once and that other simultaneous requests wait for the result instead of making redundant calls. This is where the Lock model can help. ```javascript const LockModel = this.app.getModel("Lock"); /** * Acquires a lock based on the lock name. * @param {string} name * @param {number} [ttlSeconds=30] * @returns {Promise} */ async acquireLock(name, ttlSeconds = 30) /** * Releases a lock based on the lock name. * @param {string} name * @returns {Promise} */ async releaseLock(name) /** * Waits for a lock based on the lock name. * @param {string} name * @returns {Promise} */ async waitForUnlock(name) /** * Gets the lock's remaining time based on the lock name. * @param {string} name * @returns {Promise<{ttl: number}>} */ async getLockData(name) /** * Gets the locks' remaining time based on the lock names. * @param {string[]} names * @returns {Promise<{name: string, ttl: number}[]>} */ static async getLocksData(names) ``` Example of usage: ```javascript async someHTTPRequestWithExpensiveExternalAPI(req, res) { // We have some external requests, which can be simultaneous requests from different users. const LockModel = this.app.getModel("Lock"); // Let's say it's AI processing of a video, for example. const { videoId } = req.appInfo.request; // Check if we already have it. const VideoAIModel = this.app.getModel("VideoAIModel"); const videoAI = await VideoAIModel.findOne({ videoId }); if (videoAI) { return res.json(videoAI.getPublic()); } const lockName = `video-ai-processing-${videoId}`; // We don't have that video, so let's send it for processing using a lock. const isLockAcquired = await LockModel.acquireLock(lockName); if (isLockAcquired) { const result = await videoAIService.processVideo(videoId); const videoModel = await VideoAIModel.create({ videoId, result }); // Release the lock. await LockModel.releaseLock(lockName); // Return the result. return res.json(videoModel.getPublic()); } // We don't have a lock, so let's wait for one. await LockModel.waitForUnlock(lockName); // It looks like the external process is finished, so let's check for the result. const videoAI2 = await VideoAIModel.findOne({ videoId }); if (videoAI2) { return res.json(videoAI.getPublic()); } // If there's no result, we'll return an error. return res.status(500).json({ error: "Something went wrong" }); } ``` --- # Document 6: 05-modelsOld # Models (OLD) :::warning This documentation for old model defenitions. Please do not use this type of models at all. Use ceuurnt model versions ::: Framework based on [mongoose](https://mongoosejs.com/) library and provide direct access to mongoose. It uses [ES6 class variant](https://mongoosejs.com/docs/guide.html#es6-classes) of mongoose to init. Also you can access framework Model take care about database connection :::note Models files part of [framework inheritance process ](03-files-inheritance.md). ::: ## Access mongoose instance Inside the class mongoose avaialable as ```js this.mongooseModel; ``` ## Basic model ```js import AbstractModel from "@adaptivestone/framework/modules/AbstractModel.js"; class SomeModel extends AbstractModel { constructor(app) { super(app); // you can put some init stuff } initHooks() { super.initHooks(); // place to init plugins, indexes, etc. // As it happens after loaded class into mongoos, but before mongoose inited class // this.mongooseSchema.plugin(PLUGIN_NAME); } // here mongoose scheme go // this is a fullu mongoose scheme // Please reffer to mongoose documentation https://mongoosejs.com/docs/guide.html get modelSchema() { return { someString: { type: String, required: true }, firstName: String, lastName: String, email: String, }; } // here mongoose scheme options go // this is a fullu mongoose scheme // Please reffer to mongoose documentation https://mongoosejs.com/docs/guide.html#options get modelSchemaOptions() { return { read: "primary", }; } // Static method will be part on monngose class // this.app.getModel('SomeModel').someStaticMethod() static async someStaticMethod() { const somedata = await this .findByIdAndUpdate //..... (); const { app } = this.getSuper(); return somedata; } // any methods will be part on instance method // await this.app.getModel('SomeModel').findById(124).someInstanceMethod() async someInstanceMethod() { // you can access app into the instance method const { app } = this.getSuper(); // inside of instance method you can access model data this.someString; } // any getters will became a mongoose virtual // const SomeModels = this.app.getModel("SomeModel"). // const someModelInstance = await SomeModel().create({ email: 'test@gmail.com' });; // `domain` is now a property on SomeModels documents. // someModelInstance.domain; // 'gmail.com' get domain() { return this.email.slice(this.email.indexOf("@") + 1); } // setters also be an virtual // const SomeModels = this.app.getModel("SomeModel"). // const someModelInstance = new SomeModel(); // Vanilla JavaScript assignment triggers the setter // someModelInstance.fullName = 'Jean-Luc Picard'; set fullName(v) { // `v` is the value being set, so use the value to set // `firstName` and `lastName`. const firstName = v.substring(0, v.indexOf(" ")); const lastName = v.substring(v.indexOf(" ") + 1); this.set({ firstName, lastName }); } } export default SomeModel; ``` :::tip If you have some relations ("ref") on a mongoose model that you should care to load schema. As mongoose can only build relationships with schemas in memory. Google place to do that - inside constructor. Be aware on loop linking models ```js constructoror(app){ super(app); this.app.getModel(“ReferenceModelName”); } ``` ::: :::warning Models united onse per process and then united instances cached. Do NOT expect constructor or init hook calls on every model loading ::: :::tip Please do not name the model in plural form. **Bad** - Coin**s** **Good** - Coin ::: ## API ```js getModel(modelName: string): MongooseModel; ``` Example: ```js const UserModel = this.app.getModel("User"); const userInstace = await UserModel.findOne({ email: "user@email.com" }); ``` ## Configuration Main configuration variable "MONGO_DSN" environment variable. Based on it model will made connection to the database ## Built-in models Framework came with few built-in models. ### User It's a part of the authorization system. It takes care of storing users, hash passwords and provides some basic functions for token generation and getting users. If you want to have your own user implementation you should overwrite it or disable. Auth controller depends on this model #### API ```js const UserModel = this.app.getModel("User"); const user = await UserModel.getUserByEmailAndPassword("email","password"): const userToken = await user.generateToken(); // generate and store token on databse const userPublic = await user.getPublic(); const hashedPassword = await UserModel.hashPassword("password"); const sameUser = await UserModel.getUserByToken(userToken); const sameUserAgain = await UserModel.getUserByEmail(user.email); const recoveryToken = await UserModel.generateUserPasswordRecoveryToken(user); const sameUSerAgain2 = await UserModel.getUserByPasswordRecoveryToken(recoveryToken); const isSuccess = await user.sendPasswordRecoveryEmail(i18n); const verificationToken = await UserModel.generateUserVerificationToken(user); const sameUSerAgain3 = awaitUserModel.getUserByVerificationToken(verificationToken); await UserModel.removeVerificationToken(verificationToken); const isSuccess2 = await user.sendVerificationEmail(i18n); ``` ### Migration Migration model it's helper for migration subsystems. It stores migrated files to make sure that migrated executed once Please refer to CLI/migrations for more details You probably should not use this model directly ### Sequence Sequence allows you to generate sequences by name. This is cross server safe method to generate sequences in distributed environment ```javascript const SequenceModel = this.app.getModel("Sequence"); // will be 1 const someTypeSequence = await SequenceModel.getSequence("someType"); // will be 2 const someTypeSequence2 = await SequenceModel.getSequence("someType"); // will be 1 as type is another const someAnotherTypeSequence = await SequenceModel.getSequence( "someAnotherType" ); ``` ### Lock Lock model is designed to provide ability to lock some resources in distributed environment. This can be external requests, some actions in the system, etc. Imagine that you have a log of traffic that asks external system for some data. You have cache of this data as weel, but initially you shaul asks internal api to get this data. And you want to make sure that you asking this api for that data only one time and other same time requests will wait for the result instead of asking api. This is where Lock model can help you. ```javascript const LockModel = this.app.getModel("Lock"); /** * acquire lock based on lock name * @param {string} name * @param {number} [ttlSeconds=30] * @returns {Promise} */ async acquireLock(name, ttlSeconds = 30) /** * release lock based on lock name * @param {string} name * @returns {Promise} */ async releaseLock(name) /** * wait lock based on lock name * @param {string} name * @returns {Promise} */ async waitForUnlock(name) /** * get lock remaining time based on lock name * @param {string} name * @returns {Promise<{ttl: number}>} */ async getLockData(name) /** * get lock remaining time based on lock name * @param {string[]} names * @returns {Promise<{name: string, ttl: number}[]>} */ static async getLocksData(names) ``` Example of usage: ```javascript async someHTTPRequestWithExpensiveExternalAPI(req,res){ // we have some external request (this can be same time request from different users) const LockModel = this.app.getModel("Lock"); // lets say its a AI processing of video (for example) const {videoId} = req.appInfo.request; // check if we already have it const VideoAIModel = this.app.getModel("VideoAIModel"); const videoAI = await VideoAIModel.findOne({videoId}); if(videoAI){ return res.json(videoAI.getPublic()); } const lockName = `video-ai-processing-${videoId}`; // we have no that video, lets send it otp rocessing with lock const isLockAcquired = await LockModel.acquireLock(lockName); if(isLockAcquired){ const result = await videoAIService.processVideo(videoId); const videoModel = await VideoAIModel.create({videoId, result}); // release lock await LockModel.releaseLock(lockName); // return result return res.json(videoModel.getPublic()); } // we have no lock, lets wait for it await LockModel.waitForUnlock(lockName); // lookslikeexternal process is done, lets check if we have result const videoAI2 = await VideoAIModel.findOne({videoId}); if(videoAI2){ return res.json(videoAI.getPublic()); } // we have no result, lets return error return res.status(500).json({error: "something went wrong"}); } ``` --- # Document 7: 06-Controllers > 01-intro # Controllers Controllers are a crucial component of the framework, which is built on top of the [Express.js](https://expressjs.com/) HTTP framework. They provide a convenient way to build complex systems with a robust HTTP infrastructure. :::note Controller files are part of the [framework inheritance process](03-files-inheritance.md). ::: The framework provides built-in error handling, automatic controller loading (including from subfolders), and request validation and casting. :::note In production, the framework uses the `cluster` module to start multiple instances (based on the number of CPU cores) and provide load balancing between them. Keep in mind that you cannot access one process from another. For complex scenarios (like a WebSocket server), you will need to use inter-process communication techniques to send messages between processes. ::: ## Controller Structure ```js import AbstractController from "@adaptivestone/framework/modules/AbstractController.js"; class ControllerName extends AbstractController { constructor(app, prefix) { // Optional constructor. In case you want to keep req.params from the main router. // By default, params from the parent router are omitted. // Useful when some params exist on the "getHttpPath" path. super(app, prefix, true); } get routes() { // Return routes info. // NECESSARY part. } getHttpPath() { // Return the path for Express (in 99% of cases, optional). } static get middleware() { return new Map(); // Return middlewares for THIS route only. } } export default ControllerName; ``` :::tip Only the `routes` getter is required; other parts of the controller can be omitted if not needed. ::: :::warning Controllers should extend the "AbstractController" module. ::: ## Name Convention and Loading The framework will load any file (except for `*.test.js` and `*.test.ts` files) and initialize it as an HTTP module. By default, the filename will be used as the route name. This behavior can be customized by providing your own `getHttpPath` function. For the example above: ```js class ControllerName extends AbstractController { ``` The route will be “http://localhost:3300/controllername”. Then, any method from the router will be accessible via the URL. If you want to define a custom path, you can provide your own implementation of the `getHttp-Path` function. ```js getHttpPath() { return "superDuperMegaSpecialRoute"; } ``` By default, `getHttpPath` resolves the current folder and filename and uses them to construct the route name. ## Request Flow ![RequestFlow](/img/requestFlow.jpg) ## View By default, the framework uses Express to render views with the Pug template engine. To render a view, create a template file in the `views` folder and then call it with the required parameters: ++++... [message is too long] ```js res.render("template", { title: "Hey", message: "Hello there!" }); ``` ## JSON JSON is the most common data format for modern web applications, but its flexibility can sometimes lead to confusion. We provide [basic documentation](https://andrey-systerr.notion.site/API-JSON-41f2032055ae4bddae5d033dc28eb1d3) on how we recommend working with JSON, and the framework is designed to follow these guidelines. ## Configuration The configuration file is located at `config/http.js`. Please take a moment to review it. The most notable options are: ```js port; // Port that the server will use. By default, process.env.HTTP_PORT or port 3300. hostname; // IP to bind to. By default, process.env.HTTP_HOST or '0.0.0.0' (any). Could be dangerous. corsDomains; // CORS-allowed domains. ``` --- # Document 8: 06-Controllers > 02-routes # Routes Routes are relative to the controller route. You SHOULD NOT use the full route here. Route objects have multiple levels. ## Route First Level (Method Level) On the first level, only the ‘method’ (post, put, delete, etc.) exists. Only requests with these methods will go deeper into the real routes. ```js import AbstractController from "@adaptivestone/framework/modules/AbstractController.js"; class ControllerName extends AbstractController { get routes() { return { post: { // post routes }, get: { // get routes }, put: { // put routes }, // etc. }; } } export default ControllerName; ``` ## Route Second Level (Path Level) Inside the methods (second level), we have a path. It follows the https://expressjs.com/en/guide/routing.html#route-paths Express documentation. :::tip In most cases, a few options are enough: ```js "/fullpath"; // Grab variables paramOne and paramTwo into req.params "/fullpath/:paramOne/:paramTwo"; // Like the previous one, but "paramTwo" is now optional "/fullpath/:paramOne/{:paramTwo}"; ``` ::: :::note The order of routes matters. The first matched route will be executed. ::: Example: ```js import AbstractController from "@adaptivestone/framework/modules/AbstractController.js"; class ControllerName extends AbstractController { get routes() { return { post: { "/someUrl": { handler: this.postSomeUrl, request: yup.object().shape({ count: yup.number().max(100).required(), }), }, }, }; } } export default ControllerName; ``` ## Route Third Level (Route Object Level) On the third level, we have a "route object," a special object that will describe our route. ```js { handler: this.postSomeUrl, // required request: yup.object().shape({ // optional count: yup.number().max(100).required(), }), query: yup.object().shape({ // optional page: yup.number().required(), }), middleware: [RateLimiter], // optional description: yup.string() // optional } ``` Here: ```js Handler; // Some async function (most likely in this controller file) that will do all the work. Request; // A special interface that will do validation of body parameters for you. Query; // A special interface that will do validation of query parameters for you. Middleware; // An array of middlewares specially for the current route. Description; // A description of this route (used when generating documentation). ``` ## Request Request does validation and casting of an upcoming `req.body`. As we want to use already well-defined solutions, we believe that [yup](https://github.com/jquense/yup) is a great example of how a schema should be validated. But you still have the ability to provide your own validation based on an interface. :::warning Request works on a body level. ::: Request contains all fields from `req.body` and passes them into validation. :::warning Please note that GET methods have no BODY. ::: Parameters after validation are available as `req.appInfo.request`. :::warning Do not use `req.body` directly. Always use parameters via `req.appInfo.request`. ::: ## Query Query does validation and casting of an upcoming `req.query`. The Yup schema is described similarly to the request. :::warning Query works on a query level. ::: Query contains all fields from `req.query` and passes them into validation. Parameters after validation are available as `req.appInfo.query`. :::warning Do not use `req.query` directly. Always use parameters via `req.appInfo.query`. ::: ## Yup Validation ```js request: yup.object().shape({ count: yup.number().max(100).required("error text"), }); query: yup.object().shape({ page: yup.number(), }); ``` Please follow the Yup documentation for a deeper understanding of how schemas work. All parameters are located here: [https://github.com/jquense/yup#api](https://github.com/jquense/yup#api). Example: ```js request: yup.object().shape({ name: yup.string().required("validation.name"), // You can use i18n keys here. email: yup.string().email().required("Email required field"), // Or just text. message: yup .string() .required("Message required field") .min(30, "minimum 30 chars"), // Additional validators for different types exist. pin: yup.number().integer().min(1000).max(9999).required("pin.pinProvided"), status: yup .string() .required("Status required field") .oneOf(["WAITING", "CANCELED"]), // One of. transaction: yup .object() // Deep-level object. .shape({ to: yup.string().required(), amount: yup.number().required(), coinName: yup.string().oneOf(["btc", "etc"]).default("etc"), // Default. }) .required(), }); ``` ### File Validation For file validation, we provide a special Yup class, `YupFile`. It is really simple to use. ```js import { YupFile } from "@adaptivestone/framework/helpers/yup.js"; request: yup.object().shape({ someFileName: new YupFile().required("error text"), otherFiled: yup.string().required(), // Yes, you can mix it with regular data. }); ``` :::warning Please be aware that a file can only be uploaded by ‘multipart/form-data’, and because of this, you can’t use nested objects. ::: ### Own Validation To create your own validator, your object should have two methods: ```js async validate(req.body) // Throw an error on validation failed. async cast(req.body) // Should strip unknown parameters. ``` The error object should provide an “errors” array - error (why validation failed) and a "path" string - body parameter. ```js try { await request.validate(req.body); } catch (e) { // e.path // e.errors } req.appInfo.request = request.cast; ``` ### i18n In any fields that can generate an error (required, etc.), you can use i18n keys to translate. The framework will handle the translation for you. Please refer to the [i18n documentation](08-i18n.md). ## Handler Handler - some async function (most likely in this controller file) that will do all the work. It is better to write the function in the same file. :::warning The handler can only be an **async** function. ::: `req.appInfo.app` ```js import AbstractController from "@adaptivestone/framework/modules/AbstractController.js"; class ControllerName extends AbstractController { get routes() { return { post: { '/': { handler: this.postSample, request: yup.object().shape({ count: yup.number().max(100).required(), }) } } } } // Send a request with data {count: "5000"}. // Will produce an error with status 400 and {errors: {count:['Text error']}}. postSample(req, res) { // On success validation, we pass here. // {count: "5000"} console.log(req.appInfo.request) // {count: 5000} -> casted to a number const SomeModel = this.app.getModel('SomeModel'); const SomeModelAlternativeWay = req.appInfo.app.getModel('SomeModel'); const { count } = req.appInfo.request; const someModel = await SomeModel.findOne({count}); return res.status(200).json({modelId: someModel.id}); } } export default ControllerName; ``` ### Middleware Middleware - an array of middlewares specially for the current route. :::warning Route middlewares take precedence over middlewares in controllers. ::: ```javascript import AbstractController from "@adaptivestone/framework/modules/AbstractController.js"; class ControllerName extends AbstractController { get routes() { return { get: { '/routeName': { handler: ..., middleware: [MiddlewareName, MiddlewareName, etc] } }, }; } } export default ControllerName; ``` Similarly to controller middlewares, you can use middlewares with parameters. :::note The rules for the design of middlewares with parameters are described in the subsection "Middleware". ::: Sample: ```javascript import AbstractController from "@adaptivestone/framework/modules/AbstractController.js"; import RoleMiddleware from "@adaptivestone/framework/services/http/middleware/Role.js"; class ControllerName extends AbstractController { get routes() { return { get: { '/routeName': { handler: ..., middleware: [[RoleMiddleware, { roles: ['client'] }]] } }, }; } } export default ControllerName; ``` --- # Document 9: 06-Controllers > 03-middleware # Middleware You can read more about middlewares at [https://expressjs.com/en/guide/using-middleware.html](https://expressjs.com/en/guide/using-middleware.html). In general, it’s a function that accepts a request, a response, and a `next` callback. This function can analyze requests and add more details to them (like parsing JSON, getting query params, or checking a user token). Middleware can pass requests to the next level (the next middleware or handler) or can respond directly and finish the request. This is really powerful and allows developers to reuse simple logic and build routes on these simple building blocks. Default: ```js static get middleware() { return new Map([['/{*splat}', [GetUserByToken, Auth]]]); } ``` ## Middleware Order Middleware will be executed in the order provided. Based on that, you can chain middleware where the input of the second middleware depends on the output of the first middleware. ## Global Middlewares The framework internally uses a few middlewares. These middlewares are not adjustable (for now) and are executed on each request. [Cors](#cors) [PrepareAppInfo](#prepareappinfo) [IpDetector](#ipdetector) [RequestLogger](#requestlogger) [RequestParser](#requestparser) [I18n](#i18n) ## Including Middlewares into Controllers Controller-level middleware is adjusted based on the “middleware” getter. ```js static get middleware() { return new Map([['METHOD/path', ["Middleware","Array"]]]); // return middlewares for THIS route only } ``` Where 'METHOD/path' is a method with a path. Any methods supported by the Express server are supported here (GET, POST, PUT, DELETE, etc.). It is possible to start a router with any method that is supported by Express, and the middleware will be scoped by this method. If the middleware route starts with `/`, then ALL methods will be used. The path is part of the Express regex path: [https://expressjs.com/en/5x/api.html#router.methods](https://expressjs.com/en/5x/api.html#router.methods). The middleware array is an array of middlewares (with params). Sample: ```javascript static get middleware() { return new Map([['GET/{*splat}', [GetUserByToken]]]); } ``` ```javascript static get middleware() { return new Map([ ['POST/someUrl', [ GetUserByToken, [RoleMiddleware, { roles: ['admin'] }] ]] ]); } ``` :::warning The middleware here are not raw Express middlewares. Please see below. ::: ## Including Middlewares into a Route Object Middlewares can also be added into a route object (subchapter “Routes”). ## Middleware Parameters Some middleware accept initial parameters passed into them. ```javascript static get middleware() { return new Map([ ['POST/someUrl', [ GetUserByToken, // middleware with no parameters [RoleMiddleware, { roles: ['admin'] }] // middleware with parameters ]] ]); } ``` To pass parameters, wrap the middleware in an array. The first element will be the middleware itself, and the second one will be the middleware parameters. The second one will be passed as is into the middleware constructor. ## Built-in Middleware The framework has a few middlewares that you can use. ### Auth ```js import Auth from "@adaptivestone/framework/services/http/middleware/Auth.js"; ``` Allows passing only if the user is provided. Please use any middleware that provides a user instance beforehand (like `GetUserByToken`). #### Parameters No parameters. ### Cors ```js import Cors from "@adaptivestone/framework/services/http/middleware/Cors.js"; ``` Adds CORS headers if the origin matches the config. #### Parameters `origins` - an array of strings or regex to check the origin. Required parameter. ```javascript static get middleware() { return new Map([ ['GET/someUrl', [ [Cors, { origins: ['http://localhost',/./] }] ]] ]); } ``` ### GetUserByToken ```js import GetUserByToken from "@adaptivestone/framework/services/http/middleware/GetUserByToken.js"; ``` Grabs a token and tries to parse the user from it. It will find the user in the database by the token. If the user exists, it will add the `req.appInfo.user` variable. #### Parameters No parameters. ### I18n ```js import I18n from "@adaptivestone/framework/services/http/middleware/I18n.js"; ``` An internationalization module based on [i18next](https://www.npmjs.com/package/i18next). It provides `req.appInfo.i18n` that can be used for translation. The middleware provides a few detectors: - X-Lang header - Query - User Please check the [i18n documentation](08-i18n.md) for more details. #### Parameters No parameters. ### IpDetector ```js import IpDetector from "@adaptivestone/framework/services/http/middleware/IpDetecor.js"; ``` This middleware will detect the client's IP. It works well with different proxies (AWS ELB, Nginx, etc.) and detects the real client IP. :::note If the request IP is from a `trustedProxy` (trusted source) only, then the module will try to parse the IP from the provided `X-Forwarded-For` header and grab the client IP from there. Otherwise, the request IP will be used. ::: This is a core middleware, and some other middlewares (like `RateLimiter`) depend on it. #### Parameters All parameters go into the config file. There are two parameters there: `headers` and `trustedProxy`. `headers` is an array of headers to parse the IP address from. By default, it is 'X-Forwarded-For'. `trustedProxy` is an IP, CIDR, or range of IPv4 and IPv6 that is trusted to parse headers from. ```javascript headers: ['X-Forwarded-For'], trustedProxy: [ // list of trusted proxies. '169.254.0.0/16', // ipv4 cidr 'fe80::/10', // ipv6 cidr '127.0.0.1', // ip itself '1.1.1.1-1.1.1.3', // ip range ], ``` IP data is available at: ```javascript req.appInfo.ip; ``` :::warning Select `trustedProxy` really carefully, as anyone can add any headers to your request. ::: Nginx sample to add a header: ```bash server { location xxxx/ { proxy_set_header X-Forwarded-For $remote_addr; } } ``` ### Pagination ```js import Pagination from "@adaptivestone/framework/services/http/middleware/Pagination.js"; ``` The pagination middleware provides a helper that grabs URL search parameters (`page`, `limit`) and calculates the necessary `appInfo` properties (`skip`, `limit`, and `page`). #### Parameters `limit` = 10 - default limit if not provided. `maxLimit` = 100 - max limit for documents. ```javascript static get middleware() { return new Map([ ['POST/someUrl', [ [Pagination, { limit: 10,maxLimit: 50}] ]] ]); } ``` ```javascript static get middleware() { return new Map([ ['POST/someUrl', [Pagination] ]); } ``` ```javascript // http://localhost:3300/someUrl?limit=10&page=2 const { limit, skip, page } = req.appInfo.pagination; ``` ### PrepareAppInfo ```js import PrepareAppInfo from "@adaptivestone/framework/services/http/middleware/PrepareAppInfo.js"; ``` `PrepareAppInfo` is a special small middleware that generates `res.appInfo = {}`. This is to make sure that all subsequent middleware can use `appInfo` without checking if it exists. It is for internal use. #### Parameters No parameters. ### RateLimiter A rate limiter middleware. Limits the amount of requests. ```js import RateLimiter from "@adaptivestone/framework/services/http/middleware/RateLimiter.js"; ``` For rate limiting, we are using the [node-rate-limiter-flexible](https://github.com/animir/node-rate-limiter-flexible) module. Please refer to the module documentation for more details. The basic idea of a rate limiter is that we have some weight for the call and some key that has ‘credits’. Each call consumes ‘credits’, and when it reaches 0, the request is blocked. Some samples - login protection. We can generate rate limiters based on the user's email and limit each email to only 5 calls per minute. Or we can construct a more complex login that includes user IPs, etc. #### Parameters By default, the rate key is generated based on the Route, IP, and userID. But you can adjust it via the config (globally) or via middleware parameters. ```javascript static get middleware() { return new Map([ [ 'POST/login', [ GetUserByToken, [ RateLimiter, { consumeKeyComponents: { ip: false }, limiterOptions: { points: 5 }, }, ], ], ], ]); } ``` The rate limiter middleware allows you to include request components (`req.body`) for key generation. Please note that you have no access to `req.appInfo.request` at this stage. ```javascript static get middleware() { return new Map([ ['POST/login', [ GetUserByToken, [RateLimiter,{consumeKeyComponents: { ip: false, request:['email','phone'] }}] ]] ]); } ``` You can find the default parameters in ‘config/rateLimiter.js’. These parameters are used if other parameters are not provided. The rate limiter has multiple backends (memory, Redis, and Mongo). By default, the 'memory' backend is activated. ### RequestLogger ```js import RequestLogger from "@adaptivestone/framework/services/http/middleware/RequestLogger.js"; ``` A small middleware that logs request info (route, method, status, and time). Logs example: ```js [middlewareRequestLogger] 2023-01-24T07:00:35.680Z info : Request is [GET] /project/123 [middlewareRequestLogger] 2023-01-24T07:00:35.747Z info : Finished Request is [GET] /project/123. Status: 200. Duration 67 ms ``` #### Parameters No parameters. ### RequestParser ```js import RequestParser from "@adaptivestone/framework/services/http/middleware/RequestParser.js"; ``` This is the main middleware to parse requests (`application/json`, `multipart/form-data`, `application/octet-stream`, `application/x-www-form-urlencoded`). It is based on the [formidable](https://www.npmjs.com/package/formidable) package. After parsing, the data is available in `req.body`. #### Parameters No parameters. ### Role Checks the user role (`user.roles` property). If the user does not have the role, it stops the request and returns an error. It uses OR logic (any of the specified roles will allow the user to pass). ```js import Role from "@adaptivestone/framework/services/http/middleware/Role.js"; ``` #### Parameters `roles` - an array of roles to check. It uses OR logic (any role will pass). ```javascript static get middleware() { return new Map([ ['POST/someUrl', [ [RoleMiddleware, { roles: ['admin','moderator'] }] ]] ]); } ``` ### StaticFiles :::warning Deprecated and removed in version 5. It is better to use an HTTP server (Nginx, etc.) to handle static files. ::: ```bash # nginx sample server { root /var/www/application/src/public; server_name _; client_max_body_size 64M; location / { # First attempt to serve request as file, then # as directory, then fall back to displaying a 404. try_files $uri $uri/ @backend; } location @backend { proxy_pass http://localhost:3300; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } } ``` ```js import StaticFiles from "@adaptivestone/framework/services/http/middleware/StaticFiles.js"; ``` Handles static files. Mostly for development purposes. In production, it is better to handle files via a web server. #### Parameters `folders` - an array of folders to handle files from. Required parameter. ```javascript static get middleware() { return new Map([ ['POST/someUrl', [ [StaticFiles, { folders: ['/var/www/public','/opt/public'] }] ]] ]); } ``` ## Creating Your Own Middlewares (or Integrating External Ones) You can create your own middleware. To do that, you should extend `AbstractMiddleware` and provide at least two of your own functions: `description` and `middleware`. Please check the code below. ```js import AbstractMiddleware from "@adaptivestone/framework/services/http/middleware/AbstractMiddleware.js"; class CustomMiddleware extends AbstractMiddleware { static get description() { return "Middleware description"; } // optional static get usedAuthParameters() { // Array of parameters that are used for authorization within the middleware return [ { name: "Authorization", // name of the parameter type: "apiKey", // apiKey, http, oauth2, openIdConnect in: "header", // header, query, cookie description: this?.description, }, ]; } // optional get relatedQueryParameters() { // A Yup object that defines middleware-related `req.query` parameters. It allows you to validate and get those parameters in `req.appInfo.query` relative to the route in which the middleware is declared. return yup.object().shape({ limit: yup.number(), // For example }); } // optional get relatedRequestParameters() { // A Yup object that defines middleware-related `req.body` parameters. It allows you to validate and get those parameters in `req.appInfo.request` relative to the route in which the middleware is declared. return yup.object().shape({ name: yup.string().required(), // For example }); } async middleware(req, res, next) { // check something if (!req.body.yyyyy) { // return and stop processing return res.status(400).json({}); } if (this.params.iiii) { // we can also check all the params that we passed during init } // go to the next one return next(); } } export default CustomMiddleware; ``` --- # Document 10: 07-logging # Logging The framework uses the [Winston](https://github.com/winstonjs/winston) logger as the main subsystem for logging. You can adjust different transports (console and Sentry are available by default) and add your own transport with your own parameters. The framework provides a basic initialization of Winston so you can easily use it out of the box, but you still have the ability to adjust it as you want. ## Logging Levels [RFC5424](https://datatracker.ietf.org/doc/html/rfc5424) defines logger levels. Higher levels will include messages from lower levels. In other words, if you set the ‘warn’ level, the logger will report ‘error’ as well, but will not report the ‘debug’ level. You can read more about levels in the [Winston levels documentation](https://github.com/winstonjs/winston#logging-levels). But in short, these are the default levels: ```js { error: 0, warn: 1, info: 2, http: 3, verbose: 4, debug: 5, silly: 6 } ``` ## API Each class has access to the logger instance via: ```js this.logger; ``` ```js this.logger.error("error message"); this.logger.warn("warn message"); this.logger.info("info message"); this.logger.http("http message"); this.logger.verbose("verbose message"); this.logger.debug("debug message"); this.logger.silly("silly message"); ``` :::tip Please note that `this.logger` is an instance of `winston.logger`, and you can use any methods from it. ::: ## Default Transports By default, the framework provides two transports: **console** and **Sentry** ## Configuration We try to keep the configuration simple and powerful - you are able to enable/disable loggers and add your own with all available options passed to the transport. Configuration files are located in ‘config/log.js’. ```js export default { transports: [ { transport: "sentry", // transport name (specail name or npm package name) transportOptions: { // options that will be passed to the transport instance level: process.env.LOGGER_SENTRY_LEVEL || "info", }, enable: process.env.LOGGER_SENTRY_ENABLE || false, // whether the transport is enabled or not }, { // .... }, /// more transports ], }; ``` The config contains a “transports” array. Each transport can be included in the logger. The transport **name** is an npm package name that the framework will require. You can use any transport from NPM. The **transportOptions** field contains the transport options - you can pass any options here for the transport. Please refer to the transport documentation. And finally, the **enable** field will enable/disable modules for the logger. You can check [“NODE_ENV” in the config documentation](02-configs.md#node_env) to learn more about how you can use it depending on your environment. ## Sentry transport For Sentry, we assume it is already configured in the project and we will reuse the existing setup. If Sentry is not present in the system, the framework will display a message indicating that this feature is only available when Sentry is configured. ## Add Your Own Transport Adding your own transport is a simple process. ```js npm i ${WINSTON_TRANSPORT_PACKAGE} ``` Then add it to the ‘config/log.ts’ config file. ```js export default { transports: [ { // ..... // some already existing transports }, { // .... }, { transport: "WINSTON_TRANSPORT_PACKAGE", transportOptions: { // options that will be passed to the transport instance // .... your transport options }, enable: true, }, ], }; ``` :::tip Feel free to use environment variables in your transport config as well. That simplifies working with multiple environments. ::: ## Console Output for Logger Console loggers act in a customized way to provide more verbose info. The default console output is constructed as: `(${process.pid}) ${info.label} ${info.timestamp} ${info.level} : ${info.message}` Where: “process.pid” - the PID of the Node process. Useful in a cluster environment. “info.label” - the label of the place. More info below. “info.timestamp” - the date of the event. “info.level” - the log level (“error”, ”warn”, etc.). “info.message” - the message that was passed to the logger (e.g., `this.logger.info(“this is a message”)`). ### info.label Inside the base class, we have a method [“loggerGroup”](04-base.md#api) that is used as the first part of the info message generation. This is useful for grouping messages like “controllers”, ”models”, etc. The framework uses the following groups: “command”, “connector”, “controller”, “model”, and “CLI\_”. :::tip Feel free to introduce your own groups. ::: The second part is generated by the base class function “getConstructorName”, which by default grabs the constructor name and adds it to the info string. You can overwrite this method for your classes. Example: ``` (15950) [modelCoin] 2021-10-18T11:17:54.746Z verbose : Model have no hooks ``` Here: ``` (15950) - process PID [modelCoin] - info message ("model" - group, "Coin" - model name) 2021-10-18T11:17:54.746Z - time verbose - level Model have no hooks - message ``` ### Environment Variables #### Sentry Transport **LOGGER_SENTRY_DSN** - Sentry DSN. **LOGGER_SENTRY_LEVEL** - the log level that should go into Sentry. Default: 'info'. **LOGGER_SENTRY_ENABLE** - enable or disable the Sentry logger. Default: 'false'. #### Console Transport **LOGGER_CONSOLE_LEVEL** - the log level. Default: 'silly' (includes all). **LOGGER_CONSOLE_ENABLE** - enable or disable the console logger. Default: 'true'. --- # Document 11: 08-i18n # i18n No modern app can avoid multi-language support. The framework supports internationalization out of the box. All internationalization is based on the [i18next](https://www.i18next.com/) library. ### Middlewares Framework provides I18n middleware that uses for each http requuest. ## Detectors As we are talking about languages, we need some way for the codebase to understand what language the user should use. This feature is called detectors. Order of detection: - X-Lang header - Query - User ### X-Lang Header Detector This detector will parse the “X-Lang” header on the request to detect the user's language. The frontend should add the user's language here (“en”, ”fr”, etc.), and if the backend supports it, the app will use the given user language. :::tip “xLang” is the preferred way to work with languages in the framework. ::: Example for the frontend with Fetch: ```js fetch("https://example.com/getSomething", { headers: { "X-Lang": "en", // Added language }, }); ``` ### Query Detector The query is a simple detector. Just add the 'lookupQuerystring' parameter to your query string. 'lookupQuerystring' by default is **lng**, but you can change it as you want inside the config file. ```js const res = await fetch("https://someUrl.com?lng=en"); ``` ### User Detector The user detector tries to find an authorized user (provided by middleware) and grab the ‘locale’ field from this user. ### Adding Your Own Detector At this time, you are not able to add your own detector. Please contact us if you need that option, and we will be happy to help you. ## Configuration Please look at the ‘config/i18n.js’ file for all configuration options. There are a limited number of options available. ## Language Files All files in JSON format are located in ‘src/locales/\{localeCode\}/translations.json’. :::note Backends besides files are not supported. ::: You can find detailed documentation about JSON files in the [i18next JSON documentation](https://www.i18next.com/misc/json-format). ## API The framework provides easy integration for controllers. You can grab the i18n instance with: ```js req.appInfo.i18n; req.appInfo.i18n.language; // current language req.appInfo.i18n.t("some.phrase"); // translate some phrase https://www.i18next.com/overview/api#t ``` :::tip You can pass controller ‘request’ errors as strings for internationalization. They will be processed with i18next. ```js class SomeController extends AbstractController { get routes() { return { post: { "/login": { handler: this.postLogin, request: yup.object().shape({ email: yup.string().email().required("auth.emailProvided"), // <-- look here i18n }), }, }, }; } } ``` ::: ## Direct usage. Sometimes you may want to use i18n outside of HTTP requests (such as for emails, WebSockets, etc.). For that purpose, frameworks provide an easy way to interact with i18n using the same configuration (translations, languages, etc.). App instanse ptovides lang services for you ```ts import { appInstance } from '@adaptivestone/framework/helpers/appInstance.js'; const i18nService = await appInstance.getI18nService(); const i18n = await i18nService.getI18nForLang(lang); ``` and this is preconfigured i18next instance for you --- # Document 12: 09-testsing # Testing The framework comes with [Vitest](https://vitest.dev/) support. When you name files with the `.test.(js|ts)` extension, they will be added to the tests. :::tip Please put test files near the main files that you are testing and give them the same name. If you want to test “Auth.js”, please create a file named “Auth.test.js” and put it in the same folder. ::: ## Framework (app) Instance Of course, inside a test, you need to have access to the framework instance. It will be available via appInstanceHelper ```js import { appInstance } from "@adaptivestone/framework/helpers/appInstance.js"; ``` ## Run Tests ```bash npm test ``` ## Before Scripts The test entry point is at the project level in ‘src/test/setupVite.js’. This file prepares all folder configs, requires the framework setup, and prepares the global setup for tests. The minimum Vite config file should contain: ```js test: { globalSetup: [ // This script will start Mongo (DIST in important there) 'node_modules/@adaptivestone/framework/dist/tests/globalSetupVitest', ], setupFiles: [ './src/tests/setup.ts', // this is a config files with directory location '@adaptivestone/framework/tests/setupVitest', // This is the entry point for testing from the framework './src/tests/setupHook.ts', // This is a local config file (see below) ], } ``` ### Global setup (once per test running) You are able to provide additional setup and teardown functions to global setup. Just add your implementation in an additional (or instead of) globalSetup ```js test: { globalSetup: [ 'node_modules/@adaptivestone/framework/dist/tests/globalSetupVitest', './src/tests/globalSetup.ts', // custom file for global setup and teardown ], // ... } ``` ### Custom hooks (beforeAll, etc) per test Testing helpers provide isolation of modules, and we run preparation of the framework and teardown for each test as well. ```js test: { //... setupFiles: [ './src/tests/setup.ts', // this is a config files with directory location '@adaptivestone/framework/tests/setupVitest', // This is the entry point for testing from the framework './src/tests/setupHooks.ts', // <-- we are able to provide custom logic there ], } ``` You can provide any amount of testing configs Example: ```ts ./src/tests/setupHooks.ts import { beforeAll, afterAll, beforeEach, afterEach } from "vitest"; import { createDefaultTestUser } from "./testHelpers.ts"; beforeAll(async () => { await createDefaultTestUser(); }); afterAll(async () => { // do something }); beforeEach(async () => { // do something }); afterEach(async () => { // do something }); ``` ### Default User for Testing You are able to call the creation of a default user. The user is not created by default. You should call it manually. ```js import { defaultUser, // instance of user if default user was created defaultAuthToken, // default token for auth if user was created createDefaultTestUser, // create default user and populate defaultUser and defaultAuthToken. } from "@adaptivestone/framework/tests/testHelpers.js"; const { user, token } = await createDefaultTestUser(); // defaultUser - same user // defaultAuthToken - same token ``` ## Mongo Instance As the framework is designed to work with MongoDB and provide easy integration with it, it also comes with MongoDB integration in tests. The integration is done with the help of the [MongoDbMemoryServer](https://github.com/nodkz/mongodb-memory-server) package. By default, the framework starts the Mongo memory server and stops it afterward. So you can use Mongo during your tests. ### Mongo Tests on ARM64 Machines (Docker) For ARM64, we have an interesting situation. Mongo Inc. provides binaries for Ubuntu but not for Debian, but official Node images exist for Debian but not for Ubuntu. To solve this situation, we provide our own Node Docker image based on Ubuntu. You can find it here: [ubuntu-node-docker](https://gitlab.com/adaptivestone/ubuntu-node). ## Running Tests in CI (GitLab) An important thing about testing is that tests should be executed automatically on every Git commit. That is where CI (Continuous Integration) comes in. A `.gitlab-ci.yml` sample is below: ```yaml stages: - install - checks install: stage: install image: registry.gitlab.com/adaptivestone/ubuntu-node:latest script: - node -v - npm install artifacts: paths: - node_modules/ expire_in: 2 hour codestyle: stage: checks image: registry.gitlab.com/adaptivestone/ubuntu-node:latest needs: - install dependencies: - install allow_failure: true script: - npm run codestyle tests: stage: checks image: registry.gitlab.com/adaptivestone/ubuntu-node:latest needs: - install dependencies: - install script: - npm run test ``` ## Running Tests in CI (GitHub) It is better to look at the [repo](https://github.com/adaptivestone/framework/blob/main/.github/workflows/test.yml). ```yml # yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json name: Test on: push: branches: ["*"] jobs: test: runs-on: ubuntu-latest permissions: contents: read services: redis: image: redis:latest ports: - 6379:6379 env: LOGGER_CONSOLE_LEVEL: "error" REDIS_URI: redis://localhost steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - uses: actions/setup-node@v4 with: node-version: "latest" cache: "npm" - name: npm clean install run: npm ci - name: Run Test run: npm test - name: Upload results to Codecov uses: codecov/codecov-action@v5 with: token: ${{ secrets.CODECOV_TOKEN }} ``` ## Server instance access It's possible that in testing you will need to have low-level access to the server itself. We have a helper there too. ```js import { serverInstance } from "@adaptivestone/framework/tests/testHelpers.ts"; ``` ## HTTP Endpoint Testing The framework provides a special function `getTestServerURL` to help you construct a full URL for testing. ```js import { getTestServerURL } from "@adaptivestone/framework/tests/testHelpers.js"; const url = getTestServerURL("/auth"); ``` Full example: ```js import { getTestServerURL, defaultAuthToken, } from "@adaptivestone/framework/tests/testHelpers.ts"; describe("module", () => { describe("function", () => { it("test", async () => { expect.assertions(1); const { status } = await fetch(getTestServerURL("/some/endpoint"), { method: "POST", headers: { "Content-type": "application/json", Authorization: defaultAuthToken, }, body: JSON.stringify({ // request object oneData: 1, secondDate: 2, }), }).catch(() => {}); expect(status).toBe(400); }); }); }); ``` ## Test Helpers The framework provides a set of helpers to simplify testing. ```js import { getTestServerURL, // return server URL for testing getTestServerURL('auth'); defaultUser, // instance of user if default user was created defaultAuthToken, // default token for auth if user was created serverInstance, // server instance for low level interaction createDefaultTestUser, // create default user and populate defaultUser and defaultAuthToken. setDefaultUser, // in case you want to have own user implementation setDefaultUser(yourUser) setDefaultAuthToken, // in case you want to have own user implementation setDefaultAuthToken(token) } from "@adaptivestone/framework/tests/testHelpers.js"; ``` ## Mock In most cases, your code depends on external services, but you still need to perform testing. Calling an external service for each test can be expensive and is not necessary. For this problem, Vitest provides mock options. This is when, instead of calling the real SDK of a service, you call a fake function that provides the result without API calls. [https://vitest.dev/api/vi.html#vi-mock](https://vitest.dev/api/vi.html#vi-mock) ### Mocking a Function [https://vitest.dev/api/vi.html#mocking-functions-and-objects](https://vitest.dev/api/vi.html#mocking-functions-and-objects) You can redefine an import for your own import. ```js vi.doMock("../file.js", () => ({ fileFunction: async () => ({ isSuccess: true, }), })); ``` Redefine one method in a file: ```javascript import S3 from "../S3.js"; vi.spyOn(S3, "validateCreds").mockImplementation(() => true); ``` There are many more mocking options. Please refer to the Vitest documentation for others. --- # Document 13: 10-cli # CLI The CLI (Command Line Interface) is a part of the framework system that allows you to use the full power of the framework on the command line - manipulate data, import, export, etc. The framework scans the directory for a list of commands and provides you with the ability to see what commands are available to run. :::note CLI commands are part of the [framework inheritance process](03-files-inheritance.md). ::: ## Run Command You are able to create a group of commands by putting files in a directory. You can see ‘migration’ as an example of grouping commands. ```bash # On the project level node src/cli.ts {command} # OR npm run cli ``` The command path works the same way as in controllers. Any file will be parsed as a command, and folders will group commands together. But unlike at the controller level, you are not able to change that behavior. You are able to create a group of commands by putting files in a directory. You can see ‘migration’ as an example of grouping commands. ```js commands / migration / create.ts; // migration/create command commands / migration / migrate.ts; // migration/migrate command ``` ## API You are able to run a command from any place in the framework. ```js this.app.runCliCommand(commandName: string, args: {}): Promise; ``` Where: `commandName` - the name of the command that you want to start. `args` - the arguments that you want to pass to the command. ## Creating Your Own Command All passed arguments on the command line are parsed with the help of the `parseArgs` module. ```ts import AbstractCommand from "@adaptivestone/framework/modules/AbstractCommand.js"; import type { CommandArgumentToTypes } from "@adaptivestone/framework/modules/AbstractCommand.js"; class CommandName extends AbstractCommand { static get description() { return "Some nice description of the command"; } /** * You are able to add command arguments for parsing here. * https://nodejs.org/api/util.html#utilparseargsconfig */ static get commandArguments() { return { id: { type: "string", description: "User ID to find the user", }, email: { type: "string", description: "User email to find/create the user", }, password: { type: "string", description: "New password for the user", }, roles: { type: "string", required: true, // make sure that command will ask this from user description: "User roles as a comma-separated string (--roles=user,admin,someOtherRoles)", }, update: { type: "boolean", default: false, description: "Update the user if they exist", }, } as const; // <- this is important for type generation } static isShouldInitModels = true; // Default value. Can be omitted. /** * If true, then this command will get model paths with inheritance. */ static isShouldGetModelPaths = true; /** * Return the name of the connection that you want to use. */ static getMongoConnectionName(commandName, commandArguments) { return `CLI: ${commandName} ${JSON.stringify(args)}`; } async run() { // CommandArgumentToTypes will generate types from commandArguments description const { id, email, password, roles, update } = this .args as CommandArgumentToTypes; return new Promise((resolve, reject) => {}); } } export default CommandName; ``` :::tip For boolean types, we also support negative values (without a prefix). ```js update: { type: "boolean", }, ``` ```bash node src/cli.ts ourCommand --update ``` Update Argument will by true ```bash node src/cli.ts ourCommand --no-update ``` update argumant will be false ::: ## Framework Commands The framework comes with a few built-in commands. ### Migration Migration commands allow you to migrate some data to another, or fill the database with some data. The key point here is that a migration is executed only once per file. :::tip You can use migrations for different cases, not only to modify data in the database. ::: #### Creating a Migration The migration command comes with a template to generate a migration. ```js node src/cli.ts migration/create --name={someName} ``` After creating the migration, please edit it and implement any logic that you want here. #### Applying a Migration ```js node src/cli.ts migration/migrate ``` A migration is executed in the order it was created. It is executed only once per life. ### DropIndex Sometimes it is very useful to drop indexes on already created models. For example, you created a unique index for non-required fields and missed that `null` values also should be unique per collection. The framework will take care of creating indexes based on your model on the next start, so you can drop an index and be sure that it will be recreated after. #### Run dropindex ```js node src/cli.ts dropindex --model={modelName} ``` ### SyncIndexes Synchronizes the indexes defined in the models with the real ones indexed in the database. The command will remove all indexes from the database that do not exist in the model OR have different parameters. Then it will create new indexes. :::warning This can be a dangerous command in case you have some unique index features. ::: #### Run SyncIndexes ```js node src/cli.ts SyncIndexes ``` ### OpenAPI Documentation The framework can generate documentation in the OpenAPI (Swagger) format. That is a pretty standard format for exchanging documentation. The framework only generates a JSON file and does not provide any viewer like the online version at [https://petstore.swagger.io/](https://petstore.swagger.io/) or a self-hosted version. :::tip It is a good idea to set up documentation on the CI level for the stage environment and put the JSON file in the public directory. Then you can use online viewers to check the documentation. ::: #### Run OpenAPI ```js node src/cli.ts getopenapijson --output={PATH} ``` The output is optional. Usage example: ```js node src/cli.ts getopenapijson --output='src/public/openApi.json' ``` ### CreateUser The `createuser` command creates a new user. #### Run CreateUser ```js node src/cli.ts createuser --email=somemail@gmail.com --password=somePassword --roles=user,admin,someOtherRoles ``` Only email and password are required. You are able to update a user as well. You need to specify the email or user ID to find the user and the '--update' flag to allow user updates. ### Generate Random Bytes In some cases, you need a random byte string. This command helps you to generate a random byte string. #### Run Generate Random Bytes ```js node src/cli.ts generateRandomBytes ``` or ```js npm run cli generateRandomBytes ``` ### Generate TypeScript Types A special command to generate types for TypeScript. #### Run Generate TypeScript Types ```js node src/cli.ts generateTypes ``` or ```js npm run cli generateTypes ``` --- # Document 14: 11-cache # Cache The cache subsystem is designed to store some values for quick access and a unified interface. It is useful when you have some values grabbed from an external API or some stuff that requires a lot of calculation and does not change from time to time. Caches have an expiration time, and the developer should not worry about checking it. If a value has expired or does not exist, a callback will be executed, and its return value will be used as the value to store in the cache. :::note The cache subsystem handles all values and takes care of serialization/deserialization. ::: ## API The API is simple: ```ts async getSetValue( key: string, onNotFound: () => Promise, storeTime: number, // in seconds ): Promise; ``` By default, the store time is 5 minutes. Example: ```javascript const cacheTime = 60 * 5; // seconds const someValueFromCache = await this.app.cache.getSetValue( "someKey", async () => { const someValue = await someLongAsyncOperation(); return someValue; }, cacheTime // in seconds ); ``` :::note You can request the same value multiple times, and only one callback will be executed. All other calls will be resolved as a Promise (the same promise). ```js const promiseArr = []; for (let n = 0; n < 100; n++) { promiseArr.push( this.app.cache.getSetValue( "someKey", async () => { // Will be called once! Other calls will find that "someKey" is already processing and return the same Promise. const someValue = await someLongAsyncOperation(); return someValue; }, 3600 ) ); } ``` Please note that it works that way **per process**, as checking promises happens at the process level and is not synchronized via a master process. ::: ## Configuration For now, the cache subsystem has no configuration. But please follow the Redis configuration, as the cache depends on it. --- # Document 15: 12-email # Sending Emails :::warning As of framework version 5.0, we have moved the email module to a separate package: [@adaptivestone/framework-module-email](https://www.npmjs.com/package/@adaptivestone/framework-module-email). ::: The email subsystem is based on [Nodemailer](https://github.com/nodemailer/nodemailer). In addition, we are using [Juice](https://www.npmjs.com/package/juice) to inline CSS and [html-to-text](https://www.npmjs.com/package/html-to-text) to generate text from the HTML of files. :::note Sadly, email clients are outdated and do not support a lot of web features. Some clients do not even support “style” tags. That is why all styles should be inlined. ::: ## Installation ```bash npm i @adaptivestone/framework-module-email ``` ## Templates We support two types of templates: HTML files (not templates) or Pug templates. It’s easy to add your own template language too. For each email, you should provide an HTML version, a text version (optional), and a subject. These should be provided as separate files inside the template directory. If the text version of the email is not provided, it will be generated from the HTML version by removing all HTML tags with the help of the [html-to-text](https://www.npmjs.com/package/html-to-text) package. The template directory is located at `src/services/messaging/email/templates/{templateName}` in your project. You can change it in the config. Example of a folder: ```js html.pug; // HTML markup of the email subject.pug; // Subject to generate text.pug; // Text version of the email style.css; // Styles to inline inside the HTML ``` ### Inline Images By default, the framework email module does not inline images and keeps the links as they are. But if you want to inline some images, you can use the "data-inline" attribute in the "img" tag. ```html ``` The image path is relative to your project's "src/services/messaging/email/resources" folder. :::note The best practice is to put your images on a CDN. ::: ### Template Variables Each template has these variables: - `locale` - the current locale of the request. - `t` - the translate function from i18n. Can be a dummy function if i18n is not provided. - `globalVariablesToTemplates` - from the config. - User-provided variables (see the API section). ## API ### Init Mailer ```js import Mailer from "@adaptivestone/framework-module-email"; const mail = new Mailer( this.app, "recovery", // template name { // variables for the template. These are user-provided variables. They will be merged with the default variables. oneTemplateVariable: "1", anotherTemplateVariable: "2", }, req.appInfo.i18n ); ``` Inside the template, `oneTemplateVariable` and `anotherTemplateVariable` will be available as top-level variables. ```pug p #{oneTemplateVariable} #{anotherTemplateVariable} ``` ### Send Email ```js const result = await mail.send( "some@email.com", // To "optional@from.com", // OPTIONAL. From email. If not provided, it will be grabbed from the config. {} // OPTIONAL. Any additional options for Nodemailer: https://nodemailer.com/message/ ); ``` ### Send Raw For advanced usage (your own templates, mail headers, attachments), another low-level method exists. ```js import Mailer from "@adaptivestone/framework-module-email"; const result = await Mailer.sendRaw( this.app, // framework app "to@email.com", // To "email subject", // topic "

Email html body

", // HTML body of the email "Email text body", // OPTIONAL. If not provided, it will be generated from the HTML string. "from@email.com", // OPTIONAL. From email. If not provided, it will be grabbed from the config. {} // OPTIONAL. Any additional options for Nodemailer: https://nodemailer.com/message/ ); ``` ### Render Template In some cases, you may want to render templates to a string for future usage. For example, to send an email via Gmail OAuth2 authorization on behalf of a user. ```js const { subject, text, inlinedHTML, htmlRaw } = await mail.renderTemplate(); ``` ## Configuration Please look at the ‘config/mail.ts’ file for all configuration options. ### Environment Variables Here are the most important environment variables: ```js EMAIL_HOST; // smtp.mailtrap.io by default EMAIL_PORT; // 2525 by default EMAIL_USER; EMAIL_PASSWORD; EMAIL_TRANSPORT; // smtp by default --- # Document 16: 13-deploy # Deploy There are multiple ways to deploy your app. ## On a Server This is the simplest way. There are multiple options here, but we are recommending a PM2 + Nginx setup. ### PM2 Setup You need to install PM2 and make it autoload. ```bash npm install pm2@latest -g pm2 startup ``` Then load your code to the server and go to the server directory. ```bash NODE_ENV=production pm2 start --name YOUR_APP_NAME src/index.js pm2 save ``` That is all from the Node.js side. The app is already there and listening on localhost:3300. ### Nginx Setup ``` server { root /var/www/YOUR_FOLDER/src/public; server_name YOUR_SERVER_NAME; location / { # First attempt to serve request as file, then # as directory, then fall back to displaying a 404. try_files $uri $uri/ @backend; } location @backend { proxy_pass http://localhost:3300; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_cache_bypass $http_upgrade; } } ``` ## Docker You can create a Docker image with all your data and run it on any server, including Kubernetes. The Dockerfile can look like this: ```dockerfile FROM node:latest RUN mkdir -p /opt/app && chown -R node:node /opt/app WORKDIR /opt/app COPY --chown=node:node package.json package-lock.json ./ USER node RUN npm ci COPY --chown=node:node src/ ./src/ EXPOSE 3300 CMD [ "node", "src/index.ts"] ``` To build it, use: ```bash docker build --platform --platform linux/amd64,linux/arm64 -t YOUR_REPO_NAME:TAG . ``` You can also use `buildx`: ```bash docker buildx build --platform linux/amd64,linux/arm64 -t YOUR_REPO_NAME:TAG . --push ``` Then you can run it: ```bash docker run -it -p 3300:3300 YOUR_REPO_NAME:TAG ``` --- # Document 17: 14-helpers # Helpers The framework provides some helpers to make your code easier to work with. ## App instance You can access the app instance from anywhere. ```ts import { appInstance } from "@adaptivestone/framework/helpers/appInstance.js"; ``` The app instance is the core of the framework, allowing you to retrieve models, configurations, and more. ```ts const Model = appInstance.getModel("ModelName"); const s3config = appInstance.getConfig("s3"); // etc ``` ## Redis connection A simplified way to connect to Redis. This helper loads the configuration and adds shutdown hooks. ```ts import { getRedisClient, getRedisClientSync } from '@adaptivestone/framework/helpers/redis/redisConnection.ts'; const redisClient = await getRedisClient(); const redisClientSync = await getRedisClientSync(); ``` The only difference is that `getRedisClientSync` returns the Redis client immediately and establishes the connection in the background. ---