从nodejs动态数据库连接到mongodb或mongoose

I am trying to create a multi-tenant app (saas), where each client has its own database.

我正在尝试创建一个多租户应用程序(saas)，其中每个客户端都有自己的数据库。

My situation is:

我的情况是:

I created a middleware that would determine who the client is based on a subdomain, then retrieve the client's database connection info from a general database. I don't know how to establish a connection object for this client so as to be able to use in my controllers. And should I do this in the middleware or in a controller? And if it's in the model, how do i pass the connection string and parameters (I could use session, but I don't know how to access session from within model).

我创建了一个中间件，该中间件将根据子域确定客户机是谁，然后从通用数据库检索客户机的数据库连接信息。我不知道如何为这个客户端建立一个连接对象，以便能够在我的控制器中使用。我应该在中间件还是控制器中这样做?如果在模型中，如何传递连接字符串和参数(我可以使用会话，但我不知道如何从模型中访问会话)。

How do i do the following?

我如何做以下的事情?

1. Organisation: where do I create the db connection for client dynamically?
2. 组织:如何动态地为客户端创建db连接?
3. Inject/pass connection parameters to controller or model (where connection definition is made)
4. 将连接参数注入/传递给控制器或模型(进行连接定义)
5. After dynamic connection has been made, how do i access it globally for that client?
6. 在进行了动态连接之后，如何为该客户端全局访问它?

This is an example of my middleware, and i would like to create a mongoose connection which i would like to make dynamic (pass in client's connection info):

这是我的中间件的一个例子，我想创建一个mongoose连接，我想做一个动态连接(传递客户端的连接信息):

function clientlistener() {
    return function (req, res, next) {
       console.dir('look at my sub domain  ' + req.subdomains[0]);
       // console.log(req.session.Client.name);

    if (req.session.Client && req.session.Client.name === req.subdomains[0]) {
          var options = session.Client.options;
          var url = session.Client.url
          var conn = mongoose.createConnection(url, options);
          next();
       }
    }
}

How do I access this connection object from inside the controller? Or from the model?

如何从控制器内部访问这个连接对象?还是从模型?

Thank you.

谢谢你！

2 个解决方案

#1

This is to help others who may find themselves in similar situation as I did. I hope that it could be standardized. I dont think we should have to reinvent the wheel every time someone needs to make a multi-tenant application.

这是为了帮助那些和我处境相似的人。我希望它可以标准化。我认为我们不应该在每个人需要创建多租户应用程序的时候都重新发明*。

This example describes a multi-tenant structure with each client having its own database. Like i said there might be a better way of doing this, but because i didn't get help myself, this was my solution.

这个例子描述了一个多租户结构，每个客户端都有自己的数据库。就像我说的，也许有更好的方法，但是因为我自己没有得到帮助，这就是我的解决方案。

So here are the goals this solution targets:

这就是这个解决方案的目标:

• each client is identified by subdomain e.g client1.application.com,
• 每个客户端由子域e标识。g client1.application.com,
• application checks if subdomain is valid,
• 应用程序检查子域是否有效，
• application looks up and obtains connection information (database url, credentials, etc) from master database,
• 应用程序从主数据库查找并获取连接信息(数据库url、凭据等)，
• application connects to client database ( pretty much hands over to client),
• 应用程序连接到客户端数据库(几乎是移交给客户端)，
• application takes measures to ensure integrity and resource management (e.g use the same database connection for members of the same client, rather than make new connection).
• 应用程序采取措施确保完整性和资源管理(e)。g为同一客户的成员使用相同的数据库连接，而不是建立新的连接)。

Here is the code

in your app.js file

app.js文件中

app.use(clientListener()); // checks and identify valid clients
app.use(setclientdb());// sets db for valid clients

I've created two middlewares :

我创建了两个中间件:

• clientListener - to identify the client connecting,
• 客户监听器-识别客户端连接，
• setclientdb - gets client details from Master database, after client is identified, and then establishes connection to client database.
• setclientdb -从主数据库获取客户端详细信息，在客户端被识别之后，然后建立到客户端数据库的连接。

clientListener middleware

I check who the client is by checking the subdomain from the request object. I do a bunch of checks to be sure the client is valid (I know the code is messy, and can be made cleaner). After ensuring the client is valid, I store the clients info in session. I also check that if the clients info is already stored in session, there is no need to query the database again. We just need to make sure that the request subdomain, matches that which is already stored in session.

我通过从请求对象检查子域来检查客户机是谁。我做了一些检查，以确保客户端是有效的(我知道代码是混乱的，并且可以变得更干净)。在确保客户端有效之后，我将客户端信息存储在会话中。我还检查如果客户信息已经存储在会话中，就不需要再次查询数据库。我们只需要确保请求子域与会话中存储的子域匹配。

var Clients = require('../models/clients');
var basedomain = dbConfig.baseDomain;
var allowedSubs = {'admin':true, 'www':true };
allowedSubs[basedomain] = true;
function clientlistener() {
return function(req, res, next) {
    //console.dir('look at my sub domain  ' + req.subdomains[0]);
    // console.log(req.session.Client.name);

    if( req.subdomains[0] in allowedSubs ||  typeof req.subdomains[0] === 'undefined' || req.session.Client && req.session.Client.name === req.subdomains[0] ){
        //console.dir('look at the sub domain  ' + req.subdomains[0]);
        //console.dir('testing Session ' + req.session.Client);
        console.log('did not search database for '+ req.subdomains[0]);
        //console.log(JSON.stringify(req.session.Client, null, 4));
        next();
    }
    else{

        Clients.findOne({subdomain: req.subdomains[0]}, function (err, client) {
            if(!err){
                if(!client){
                    //res.send(client);
                    res.send(403, 'Sorry! you cant see that.');
                }
                else{
                    console.log('searched database for '+ req.subdomains[0]);
                    //console.log(JSON.stringify(client, null, 4));
                    //console.log(client);
                   // req.session.tester = "moyo cow";
                    req.session.Client = client;
                    return next();

                }
            }
            else{
                console.log(err);
                return next(err)
            }

        });
    }

   }
 }

module.exports = clientlistener;

setclientdb middleware:

I check everything again making sure that the client is valid. Then the connection to the client's database with the info retrieved from session is opened.

我再检查一遍，确保客户是有效的。然后打开与从会话检索到的信息的客户端数据库的连接。

I also make sure to store all active connections into a global object, so as to prevent new connections to the database upon each request(we don't want to overload each clients mongodb server with connections).

我还确保将所有活动连接存储到一个全局对象中，以便在每次请求时防止对数据库的新连接(我们不想让每个客户端使用连接的mongodb服务器超载)。

var mongoose = require('mongoose');
//var dynamicConnection = require('../models/dynamicMongoose');
function setclientdb() {
    return function(req, res, next){
        //check if client has an existing db connection                                                               /*** Check if client db is connected and pooled *****/
    if(/*typeof global.App.clientdbconn === 'undefined' && */ typeof(req.session.Client) !== 'undefined' && global.App.clients[req.session.Client.name] !== req.subdomains[0])
    {
        //check if client session, matches current client if it matches, establish new connection for client
        if(req.session.Client && req.session.Client.name === req.subdomains[0] )
        {
            console.log('setting db for client ' + req.subdomains[0]+ ' and '+ req.session.Client.dbUrl);
            client = mongoose.createConnection(req.session.Client.dbUrl /*, dbconfigoptions*/);


            client.on('connected', function () {
                console.log('Mongoose default connection open to  ' + req.session.Client.name);
            });
            // When the connection is disconnected
            client.on('disconnected', function () {
                console.log('Mongoose '+ req.session.Client.name +' connection disconnected');
            });

            // If the Node process ends, close the Mongoose connection
            process.on('SIGINT', function() {
                client.close(function () {
                    console.log(req.session.Client.name +' connection disconnected through app termination');
                    process.exit(0);
                });
            });

            //If pool has not been created, create it and Add new connection to the pool and set it as active connection

            if(typeof(global.App.clients) === 'undefined' || typeof(global.App.clients[req.session.Client.name]) === 'undefined' && typeof(global.App.clientdbconn[req.session.Client.name]) === 'undefined')
            {
                clientname = req.session.Client.name;
                global.App.clients[clientname] = req.session.Client.name;// Store name of client in the global clients array
                activedb = global.App.clientdbconn[clientname] = client; //Store connection in the global connection array
                console.log('I am now in the list of active clients  ' + global.App.clients[clientname]);
            }
            global.App.activdb = activedb;
            console.log('client connection established, and saved ' + req.session.Client.name);
            next();
        }
        //if current client, does not match session client, then do not establish connection
        else
        {
            delete req.session.Client;
            client = false;
            next();
        }
    }
    else
    {
        if(typeof(req.session.Client) === 'undefined')
        {
           next();
        }
        //if client already has a connection make it active
        else{
            global.App.activdb = global.App.clientdbconn[req.session.Client.name];
            console.log('did not make new connection for ' + req.session.Client.name);
            return next();
        }

    }
    }
}

module.exports = setclientdb;

Last but not the least

Since I am using a combination of mongoose and native mongo, We have to compile our models at run time. Please see below

由于我使用的是mongoose和本地mongo的组合，所以我们必须在运行时编译模型。请见下文

Add this to your app.js

将其添加到app.js中

// require your models directory
var models = require('./models');

// Create models using mongoose connection for use in controllers
app.use(function db(req, res, next) {
    req.db = {
        User: global.App.activdb.model('User', models.agency_user, 'users')
        //Post: global.App.activdb.model('Post', models.Post, 'posts')
    };
    return next();
});

Explanation:

解释:

Like I said earlier on I created a global object to store the active database connection object: global.App.activdb

如前所述，我创建了一个全局对象来存储活动数据库连接对象:global. app .activdb

Then I use this connection object to create (compile) mongoose model, after i store it in the db property of the req object: req.db. I do this so that i can access my models in my controller like this for example.

然后我使用这个连接对象来创建(编译)mongoose模型，在我将它存储到req对象的db属性:req.db之后。我这样做是为了能够像这样访问我的控制器中的模型。

Example of my Users controller:

我的用户控制器示例:

exports.list = function (req, res) {
    req.db.User.find(function (err, users) {

        res.send("respond with a resource" + users + 'and connections  ' + JSON.stringify(global.App.clients, null, 4));
        console.log('Worker ' + cluster.worker.id + ' running!');
    });

};

I will come back and clean this up eventually. If anyone wants to help me, that be nice.

我最终会回来把它清理干净的。如果有人想帮助我，那就太好了。

#2

Hello everyone, here is a much more updated solution.

So here are the goals this solution targets:

这就是这个解决方案的目标:

• each client is identified by subdomain e.g client1.application.com,
• 每个客户端由子域e标识。g client1.application.com,
• application checks if subdomain is valid,
• 应用程序检查子域是否有效，
• application looks up and obtains connection information (database url, credentials, etc) from master database,
• 应用程序从主数据库查找并获取连接信息(数据库url、凭据等)，
• application connects to client database ( pretty much hands over to client),
• 应用程序连接到客户端数据库(几乎是移交给客户端)，
• application takes measures to ensure integrity and resource management (e.g use the same database connection for members of the same client, rather than make new connection).
• 应用程序采取措施确保完整性和资源管理(e)。g为同一客户的成员使用相同的数据库连接，而不是建立新的连接)。

updates

• use of promises,
• 使用承诺,
• automatic import & compilation of models
• 自动导入和编译模型
• New middleware ; modelsinit (used to automatically import and compile mongoose models)
• 新的中间件;modelsinit(用于自动导入和编译mongoose模型)
• Clean up of middlewares (setclientdb, clientlistener, modelsInit)
• 清理中间件(setclientdb、clientlistener、modelsInit)

Please see below for some Explanations

**

* *

modelsInit Middleware

** features

* *功能

• tests if models are already compiled. If so, skip.
• 测试模型是否已经编译。如果是这样,跳过。

• tests to see if request is not a tenant request; i.e (request to apps homepage, admin page, etc)

  检查请求是否不是租户请求;我。e(请求应用程序主页、管理页面等)

  'use strict';
  /**
   * Created by moyofalaye on 3/17/14.
   */
  
  var path = require('path');
  var config = require('../../config/config');
  
  // Globbing model files
  
  config.getGlobbedFiles('./app/models/*.js').forEach(function (modelPath) {
      require(path.resolve(modelPath));
  
  });
  
  
  
  function modelsInit() {
      return function (req, res, next) {
  
  //console.log(req.subdomains[0]);
          switch (req.subdomains[0]) {
              case 'www':
              case undefined:
                  return next();
                  break;
              case 'admin':
                  return next();
                  break;
  //            default:
  //              return
          }
          var clientname = req.session.Client.name;
  
      // test if models are not already compiled if so, skip
      if (/*typeof req.db === 'undefined' && */ typeof global.App.clientModel[clientname] === 'undefined') {
          req.db = {};
       //Get files from models directory
              config.getGlobbedFiles('./app/models/clientmodels/**/*.js').forEach(function (modelPath) {
              console.log('the filepath is ' + modelPath);
              //Deduce/ extrapulate model names from the file names
              //Im not very good with regxp but this is what i had to do, to get the names from the filename e.g users.server.models.js (this is my naming convention, so as not to get confused with server side models and client side models
  
              var filename = modelPath.replace(/^.*[\\\/]/, '');
              var fullname = filename.substr(0, filename.lastIndexOf('.'));
              var endname = fullname.indexOf('.');
              var name = fullname.substr(0, endname);
              req.db[name] = require(path.resolve(modelPath))(global.App.activdb);
              console.log('the filename is ' + name);
          });
  
          global.App.clientModel[clientname] = req.db;
  
          console.log(global.App.clients);
  
          return next();
      }
      // since models exist, pass it to request.db for easy consumption in controllers
      req.db = global.App.clientModel[clientname];
      return next();
      };
  }
  
  module.exports = modelsInit;
  

Todo: Further Explanation

待办事项:进一步解释

ClientListener.js

var config = require('../../config/config');
var Clients = require('../models/clients');
var basedomain = config.baseDomain;
var allowedSubs = {'admin': true, 'www': true};
allowedSubs[basedomain] = true;

//console.dir(allowedSubs);

function clientlistener() {
    return function (req, res, next) {
        //check if client has already been recognized
        if (req.subdomains[0] in allowedSubs || typeof req.subdomains[0] == 'undefined' || req.session.Client && req.session.Client.name === req.subdomains[0]) {
            console.log('did not search database for ' + req.subdomains[0]);
            //console.log(JSON.stringify(req.session.Client, null, 4));
            return next();
        }

        //look for client in database
        else {

            Clients.findOne({subdomain: req.subdomains[0]}, function (err, client) {
                if (!err) {
                    //if client not found
                    if (!client) {
                        //res.send(client);
                        res.status(403).send('Sorry! you cant see that.');
                        console.log(client);
                    }
                    // client found, create session and add client
                    else {
                        console.log('searched database for ' + req.subdomains[0]);
                        req.session.Client = client;
                        return next();
                    }
                }
                else {
                    console.log(err);
                    return next(err)
                }

            });
        }

    }
}

module.exports = clientlistener;

setclientdb.js

var client;
            var clientname;
            var activedb;

            var Promise = require("bluebird");
            Promise.promisifyAll(require("mongoose"));
            //mongoose = require('mongoose');


            function setclientdb() {
                return function (req, res, next) {
                    //check if client is not valid
                    if (typeof(req.session.Client) === 'undefined' || req.session.Client && req.session.Client.name !== req.subdomains[0]) {
                        delete req.session.Client;
                        client = false;
                        return next();
                    }
                    //if client already has an existing connection make it active
                    else if (global.App.clients.indexOf(req.session.Client.name) > -1) {
                        global.App.activdb = global.App.clientdbconn[req.session.Client.name]; //global.App.clientdbconnection is an array of or established connections
                        console.log('did not make new connection for ' + req.session.Client.name);
                        return next();
                    }
                    //make new db connection
                    else {
                        console.log('setting db for client ' + req.subdomains[0] + ' and ' + req.session.Client.dbUrl);
                        client = mongoose.createConnection(req.session.Client.dbUrl /*, dbconfigoptions*/);
                        client.on('connected', function () {
                            console.log('Mongoose default connection open to  ' + req.session.Client.name);
                            //If pool has not been created, create it and Add new connection to the pool and set it as active connection
                            if (typeof(global.App.clients) === 'undefined' || typeof(global.App.clients[req.session.Client.name]) === 'undefined' && typeof(global.App.clientdbconn[req.session.Client.name]) === 'undefined') {
                                clientname = req.session.Client.name;
                                global.App.clients.push(req.session.Client.name);// Store name of client in the global clients array
                                activedb = global.App.clientdbconn[clientname] = client; //Store connection in the global connection array and set it as the current active database
                                console.log('I am now in the list of active clients  ' + global.App.clients[clientname]);
                                global.App.activdb = activedb;
                                console.log('client connection established, and saved ' + req.session.Client.name);
                                return next();
                            }
                        });
                        // When the connection is disconnected
                        client.on('disconnected', function () {
                            console.log('Mongoose ' + req.session.Client.name + ' connection disconnected');
                        });

                        // If the Node process ends, close the Mongoose connection
                        process.on('SIGINT', function () {
                            client.close(function () {
                                console.log(req.session.Client.name + ' connection disconnected through app termination');
                                process.exit(0);
                            });
                        });
                    }


                }
            }

            module.exports = setclientdb;

Further Explanations Coming