淘宝开源定时任务调度框架

下载TBSchedule源码svn地址：code.taobao.org/p/tbschedule/src/trunk/

内容包括两部分：TBSchedule源码及开发依赖包tbschedule、调度控制台ScheduleConsole.war(配置监控查看运行状况)

TBSchedule介绍：

1）调度机的高可用有保障，多调度机向注册中心注册后，共享调度任务，且同一调度任务仅由一台调度机执行调度，当前调度机异常宕机后，其余的调度机会接上。

2）执行机的高可用有保障，多执行机向注册中心注册后，配置执行机单线程（多机总线程为1）执行任务，调度机会随机启动一台执行机执行，当前执行异常机宕机后，调度机会会新调度一台执行机。

3）执行机的并行高效保障，配置执行机多线程且划分多任务子项后，各任务子项均衡分配到所有执行机，各执行机均执行，多线程数据一致性协调由任务项参数区分。

4）弹性扩展失效转移保障，运行中的执行机宕机，或新增执行机，调度机将在下次任务执行前重新分配任务项，不影响正常执行机任务（崩溃的执行机当前任务处理失效）；运行中的调度机宕机或动态新增调度机，不影响执行机当前任务，调度机宕机后动态切换。

TBSchedule特性【网摘】

1）TBSchedule的目的是让一种批量任务或者不断变化的任务，能够被动态的分配到多个主机的JVM中，不同的线程组中并行执行。所有的任务能够被不重复，不遗漏的快速理。

2）调度的Manager可以动态的随意增加和停止。

3）可以通过JMX控制调度服务的创建和停止。

4）可以指定调度的时间区间。

几款定时任务框架对比【网摘】

1）Quartz：Java事实上的定时任务标准。但Quartz关注点在于定时任务而非数据，并无一套根据数据处理而定制化的流程。虽然Quartz可以基于数据库实现作业的高可用，但缺少分布式并行执行作业的功能。

2）Crontab：Linux系统级的定时任务执行器，缺乏分布式和集中管理功能。

3）elastic-job：当当网最近开源项目，功能跟TBSchedule几乎一样（*TBSchedule文档缺失严重），一台服务器只能开启一个任务实例，基于Ip不基于IpPort，单机难调试集群功能。

4）TBSchedule：淘宝早期开源，稳定性可以保证。

Tbs执行流程图(来源于交流群)

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

Tbs客户端运行实例

1、 zookeeper集群搭建（windows环境下），单击模拟集群

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

创建ZookeeperLab文件夹，下面创建三个文件server1、server2、server3模拟三个zk端。

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

每个服务端下三个文件夹，data、logs、下载的zk版本解压

三个zkserver下区别

(1) logs为空即可

(2) data下放一个myid，没有扩展名哦，server1-data下myid写1，server2下写2，server3下写3，代表各自serverid，后面配置里会用到。

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

（3）修改或增加 datadir，datalogdir，为具体路径，注意server1/2/3写正确，clienport=2181/2182/2183 每个server下端口不能重复哦，底部增加三个server的端口信息，server.A=127.0.0.1:B:C,A=myid里写的数字，B=集群成员的信息交换端口，C=在leader挂掉时专门用来进行选举leader所用

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

（4）运行zk集群

分别运行server1/2/3下的zkServer.cmd,运行server1、server2的时候会出错误信息，不用管它，是因为集群没有全部启动的原因，启动server3后正常通信。

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

2、创建java job demo，并添加spring依赖

Myeclipse新建maven项目（提前部署好maven环境，此内容略，可参考https://my.oschina.net/softwarechina/blog/219633）

3、Task设计（网摘）

1、假设场景：任务需要将订单表tbOrder中制单日期在20141201--20141208內（共8天）的数据同步到备份tbOrder_copy表，其中每2天分为一个任务组并行同步（每次提取500条），关于任务组的划分和TbSchedule中TaskItem的相关概念请先参考wiki，后续也会有部分解释。

2、数据环境使用MySQL数据库，创建tbOrder和tbOrder_Copy数据表，结构相同，同时在tbOrder事先生成好测试数据，建议每天的数据量在1000条以上。

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

3、DataSyncABean

packageTask;

importjava.sql.Connection;

importjava.sql.PreparedStatement;

importjava.sql.ResultSet;

importjava.util.ArrayList;

import java.util.Comparator;

importjava.util.List;

importorg.apache.log4j.Logger;

importutil.ConnectionUtil;

importcom.taobao.pamirs.schedule.IScheduleTaskDealSingle;

importcom.taobao.pamirs.schedule.TaskItemDefine;

publicclass DataSyncABean implements IScheduleTaskDealSingle<OrderInfo> {

private static final Logger LOG =Logger.getLogger(DataSyncABean.class);

public List<OrderInfo>selectTasks(String taskParameter, String ownSign,

int taskItemNum,List<TaskItemDefine> queryCondition,

int eachFetchDataNum) throwsException {

if (queryCondition.size() == 0) {

return null;

}

StringBuffer condition = newStringBuffer();

for (int i = 0; i <queryCondition.size(); i++) {

if (i > 0) {

condition.append(",");

}

condition.append(queryCondition.get(i).getTaskItemId());

}

String sql = "select * fromtbOrder " + "where "

+ " BillNumber not in(select BillNumber from tbOrder_copy) "

+ " and RIGHT(BuildDate,1)in (" + condition + ") " + "limit "

+ eachFetchDataNum;

LOG.info(sql);

Connection connection = null;

PreparedStatement pStatement = null;

ResultSet rs = null;

List<OrderInfo> Orderlist = null;

try {

connection =ConnectionUtil.getConnection();

pStatement =connection.prepareStatement(sql);

rs = pStatement.executeQuery();

Orderlist = newArrayList<OrderInfo>();

while (rs.next()) {

OrderInfoorder = new OrderInfo();

order.BillNumber= rs.getString("BillNumber");

order.BuildDate =rs.getString("BuildDate");

order.Customer =rs.getString("Customer");

order.GoodsName =rs.getString("GoodsName");

order.Amount =rs.getFloat("Amount");

order.SaleMoney =rs.getFloat("SaleMoney");

Orderlist.add(order);

}

} catch (Exception e) {

e.printStackTrace();

return null;

} finally {

ConnectionUtil.closeResultSet(rs);

ConnectionUtil.closePreparedStatement(pStatement);

ConnectionUtil.closeConnection(connection);

}

return Orderlist;

}

public Comparator<OrderInfo>getComparator() {

return null;

}

public boolean execute(OrderInfo task,String ownSign) throws Exception {

String sql = "insert into tbOrder_copyvalues('" + task.BillNumber

+ "','" +task.BuildDate + "','" + task.Customer + "','"

+ task.GoodsName +"'," + task.Amount + "," + task.SaleMoney

+ ")";

Connection connection = null;

PreparedStatement pStatement = null;

try {

connection =ConnectionUtil.getConnection();

pStatement =connection.prepareStatement(sql);

int count =pStatement.executeUpdate();

return count > 0 ? true :false;

} catch (Exception e) {

e.printStackTrace();

return false;

} finally {

ConnectionUtil.closePreparedStatement(pStatement);

ConnectionUtil.closeConnection(connection);

}

public void aaa()

{

Stringa="df";

a="d";

}
}

3、 OrderInfo

package Task;

public classOrderInfo {

public String getBillNumber(){

returnBillNumber;

}

public void setBillNumber(StringbillNumber) {

BillNumber = billNumber;

}

public String getBuildDate() {

returnBuildDate;

}

public void setBuildDate(StringbuildDate) {

BuildDate = buildDate;

}

public String getCustomer() {

returnCustomer;

}

public void setCustomer(Stringcustomer) {

Customer = customer;

}

public String getGoodsName() {

returnGoodsName;

}

public void setGoodsName(StringgoodsName) {

GoodsName = goodsName;

}

public float getAmount() {

returnAmount;

}

public void setAmount(float amount) {

Amount = amount;

}

public float getSaleMoney() {

returnSaleMoney;

}

public void setSaleMoney(float saleMoney) {

SaleMoney = saleMoney;

}

public StringBillNumber;

public StringBuildDate;

public StringCustomer;

public StringGoodsName;

public float Amount;

public float SaleMoney;

}

4、完整pom

<projectxmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0http://maven.apache.org/xsd/maven-4.0.0.xsd">

<version>0.0.1-SNAPSHOT</version>

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

</properties>

<groupId>org.glassfish</groupId>

<artifactId>bean-validator</artifactId>

<version>3.0-JBoss-4.0.2</version>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.annotation</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.ejb</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.enterprise.deploy</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.jms</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.management.j2ee</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.eclipse.persistence</groupId>

<artifactId>javax.persistence</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.resource</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.security.auth.message</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.security.jacc</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.servlet</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.servlet.jsp</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.servlet.jsp.jstl</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish</groupId>

<artifactId>javax.transaction</artifactId>

<scope>provided</scope>

</dependency>

<groupId>javax.xml.bind</groupId>

<scope>provided</scope>

</dependency>

<groupId>javax.ws.rs</groupId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish.web</groupId>

<scope>provided</scope>

</dependency>

<groupId>javax.mail</groupId>

<scope>provided</scope>

</dependency>

<groupId>javax.xml</groupId>

<artifactId>webservices-api-osgi</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.jboss.weld</groupId>

<artifactId>weld-osgi-bundle</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.glassfish.web</groupId>

<artifactId>javax.servlet.jsp.jstl</artifactId>

</dependency>

<groupId>javax.servlet.jsp.jstl</groupId>

<artifactId>javax.servlet.jsp.jstl-api</artifactId>

</dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-core</artifactId>

</dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-beans</artifactId>

</dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-context</artifactId>

</dependency>

<groupId>commons-attributes</groupId>

<artifactId>commons-attributes-api</artifactId>

</dependency>

<groupId>commons-attributes</groupId>

<artifactId>commons-attributes-compiler</artifactId>

</dependency>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-acl</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-cas-client</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-core</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-core-tiger</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-ntlm</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-openid</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-portlet</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-taglibs</artifactId>

<version>2.0.5.RELEASE</version>

</dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-portlet</artifactId>

</dependency>

<groupId>org.samba.jcifs</groupId>

<artifactId>jcifs</artifactId>

</dependency>

<groupId>org.jasig.cas</groupId>

<artifactId>cas-client-core</artifactId>

</dependency>

<groupId>org.springframework.ldap</groupId>

<artifactId>spring-ldap-core</artifactId>

<version>1.3.0.RELEASE</version>

</dependency>

<groupId>org.springframework.webflow</groupId>

<artifactId>spring-webflow</artifactId>

<version>2.0.8.RELEASE</version>

</dependency>

<groupId>org.springframework.webflow</groupId>

<artifactId>spring-binding</artifactId>

<version>2.0.8.RELEASE</version>

</dependency>

<groupId>org.springframework.webflow</groupId>

<artifactId>spring-js</artifactId>

<version>2.0.8.RELEASE</version>

</dependency>

</dependency>

<groupId>org.springframework.javaconfig</groupId>

<artifactId>spring-javaconfig</artifactId>

</dependency>

<groupId>org.apache.zookeeper</groupId>

<artifactId>zookeeper</artifactId>

</dependency>

<groupId>com.taobao.pamirs.schedule</groupId>

<artifactId>tbschedule</artifactId>

</dependency>

<groupId>mysql</groupId>

<artifactId>mysql-connector-java</artifactId>

</dependency>

</dependency>

</dependency>

</dependency>

</dependencies>

<build>

<artifactId>maven-compiler-plugin</artifactId>

</configuration>

</plugin>

<artifactId>maven-war-plugin</artifactId>

<failOnMissingWebXml>false</failOnMissingWebXml>

</configuration>

</plugin>

<groupId>org.apache.tomcat.maven</groupId>

<artifactId>tomcat7-maven-plugin</artifactId>

<id>tomcat-deploy</id>

<phase>deploy</phase>

<goals>

<goal>deploy</goal>

</goals>

</execution>

</executions>

<url>http://localhost:8081/manager/text</url>

<username>tomcat</username>

<password>tomcat</password>

<server>tomcat7</server>

</configuration>

</plugin>

</plugins>

</build>

</project>

5、TaskCenter启动

public classTaskCenter {

private staticfinalLogger LOG= Logger.getLogger(testlog4j.class);

public static void main(String[] args)throws Exception {

// 初始化Spring

ApplicationContext ctx = newFileSystemXmlApplicationContext(

"src\\main\\java\\applicationContext.xml");

// 初始化调度工厂

TBScheduleManagerFactoryscheduleManagerFactory = new TBScheduleManagerFactory();

Properties p = new Properties();

p.put("zkConnectString","127.0.0.1:2181");

p.put("rootPath", "/taobao-pamirs-schedule/test1");

p.put("zkSessionTimeout","60000");

p.put("userName", "admin");

p.put("password", "123456");

p.put("isCheckParentPath","true");

scheduleManagerFactory.setApplicationContext(ctx);

System.out.println("---------------------------------");

LOG.info("---------------maven test------------------");

scheduleManagerFactory.init(p);

System.out.println("hello");

}

运行结果

以上内容word文档下载：http://download.csdn.net/detail/mingtianmeihao/9688421

Demo源码下载地址：http://download.csdn.net/detail/mingtianmeihao/9687371

秒客网

TBSchedule淘宝开源定时任务调度框架（附客户端源码demo）

3、Task设计（网摘）

相关文章