EX-10.4 MASTER SERVICE AGREEMENT
Published on January 16, 2008
Exhibit 10.4
MASTER SERVICES AGREEMENT
This MASTER SERVICES AGREEMENT ("Agreement") is entered into and effective as of
May 16, 2003, by and between (CHINESE CHARACTERS) MICROSOFT (CHINA) CO., LTD.
("MCCL"), a Chinese corporation having its principal place of business at 6F,
Beijing Sigma Center, No. 49, Zhichun Road, Haidian District, Beijing 100080,
P.R.C., and (CHINESE CHARACTERS) ATA TESTING AUTHORITY, BEIJING CO., LTD.
("ATA"), a Chinese corporation located Bei Hai Wan Tai Building Room 707, No.6
North Chao Yang Men Street, Beijing P.R.China 100027
RECITALS
A. MCCL has obtained proper authorization with regard to certain courseware
and exams, and has been authorized to use such courseware and exams to educate
and test end users' skill levels in various areas through its Microsoft Official
Curriculum and Microsoft Certified Professional (MCP) Program.
B. ATA has been appointed by the Ministry of Education ("MOE") to assist in
implementation of MOE Polytechnic Program (FYB011449), including curriculum
planning and distribution, school recruitment and marketing, exam delivery and
related services at test centers in China.
C. ATA wishes to be licensed by MCCL for Microsoft courseware and exams and
support the MOE Polytechnic Program, which is the degree program registered by
the Polytechnics Division of the Ministry of Education and the students eligible
under this Agreement (including the relevant Exhibits attached hereto) shall be
the fulltime students of the MOE Polytechnic Program.
D. MCCL wishes to engage ATA to deliver Microsoft's curriculum and tests to
students enrolled in the MOE Polytechnic Program through ATA's network of secure
testing centers.
In consideration of the covenants and conditions set forth below, the adequacy
of which is agreed to and hereby acknowledged, the parties agree as follows:
AGREEMENT
1. ATA Obligations.
1.1 Duties. ATA agrees to perform its duties described in the Statement of
Work ("Statement of Work"), which is attached hereto as Exhibit A, and
incorporated herein by reference. ATA understands that it shall deliver the
courseware and exam in accordance with this Agreement and the Exhibits attached
hereto only to full-time students of MOE Polytechnic Program.
1.2 Financial Information. Within ten (10) days after ATA learns that it
has become or will become insolvent, ATA shall submit financial statements to
MCCL in sufficient detail to allow MCCL to determine whether ATA shall be
capable of continuing to perform its obligations hereunder. The financial
statements shall include, but shall not be limited to, balance sheets and
related statements of income and retained earnings and statements of changes in
financial condition. To the extent those statements are audited, the audit
report of the certified public accountant performing the audit shall also be
submitted to MCCL. The financial information provided hereunder shall be deemed
to be Confidential Information pursuant to the Section 5 herein.
1.3 Reports. ATA shall timely provide MCCL with the reports specified in
the Statement of Work and the general financial reports identified in the
Statement of Work (collectively referred to as "Report"), and all other
information requested pursuant to Section 17 of the Statement of Work. All
Reports shall be complete and accurate. Each Report, whether in electronic or
paper format, shall meet the
Page 1 of 13
standard Report Requirements identified for the Report in the Statement of Work
and shall be delivered within the time specified in the Statement of Work. ATA
shall use its best effort to correct any errors in a Report within three
Business Days following MCCL's notice specifying the item in respect of which an
error may have occurred. ATA shall deliver each Report, and all supporting
documentation therefore, by the time and on the Business Day specified in the
Statement of Work. A "Business Day" means each day on which ATA or MCCL is open
for business.
1.4 Training Agreements. ATA agrees that it shall enter into a training and
testing agreement with each of the schools that enroll in the MOE Polytechnic
Program in the form and substance of Exhibit B attached hereto (the "Training
Agreement"), and shall perform its obligations in accordance with the Training
Agreement.
1.5 Printing and Distribution. ATA agrees that is shall enter into printing
and distribution agreements with publisher(s) necessary to print and distribute
courseware in accordance with Exhibit A attached hereto, and shall perform its
obligations in accordance with those agreements.
1.6 Licensing Fees. Pursuant to the Training Agreement, ATA will charge and
collect fees from enrolled schools for each enrolled student of the MOE
Polytechnic Program and shall pay MCCL licensing fees as defined in the
Statement of Work for each fulltime student that enrolls in this MOE Polytechnic
Program ("Licensing Fees")
2. MCCL Obligations.
2.1 All Exams and Courseware. MCCL will license to ATA the right to deliver
the MCP exam files and courseware in accordance with this Agreement and the
Exhibits attached hereto, and make available to ATA specifications of all
materials as listed in the Statement of Work, as updated by MCCL from time to
time in its sole discretion, to allow ATA to perform the Services.
2.2 MCCL Certification Database. MCCL will maintain and operate a candidate
certification database and provide specifications in accordance with which ATA
shall perform candidate registration and deliver test results to MCCL in
accordance with the Statement of Work.
2.3 Program Promotion. MCCL will use commercially reasonable efforts to
promote the program and ATA as an exam delivery service provider, using methods
for creating awareness selected by MCCL in its sole discretion.
2.4 Notice of Change. MCCL will notify ATA not less than sixty (60) days in
advance of authorizing an additional exam delivery partner (including test
delivery service) for the MOE Polytechnic Program.
3. Remedies.
If one party does not perform its material obligations in accordance with this
Agreement ("Responsible Party"), the other party, at its sole discretion, may
elect one or more of the following remedies (i) require that the Responsible
Party provide a written plan within fifteen (15) days of notice, describing the
cause of failure to perform, and a timetable and strategy for compliance in the
future, (ii) require that the Responsible Party implement its plan for
compliance in the future as approved by the other party, and/or (iii) terminate
the Agreement and its Exhibits in accordance with Section 10.4 hereinafter. Any
remedies under this Section 3 shall be in addition to all other remedies
available under this Agreement.
In addition to the foregoing, if ATA performs its obligations in a manner which
constitutes a failure to meet or exceed Key Performance Indicators ("KPI", as
defined in Appendix A of the Statement of Work), excluding the KPI for
Abandonment Rate, or ATA fails to timely provide complete reporting as required
in the Statement of Work, MCCL may charge ATA a penalty of five percent (5%) of
the total Licensing Fees due for the first three months after 30 days written
notice from MCCL to ATA describing the failure and
Page 2 of 13
that this remedy is to be applied and increasing two percent (2%) per month each
month that ATA continues to fail to meet the performance described in MCCL's
notice herein, up to a maximum of ten percent (25%).
4. Intellectual Property Rights.
4.1 Ownership of Exams. MCCL and its suppliers retain all right, title and
interest in the courseware and exams provided by MCCL to ATA to enable it to
perform its obligations.
4.2 Trademarks. ATA may refer to itself as a "Microsoft Certified
Professional Exam Provider" in text, and use the "Microsoft Certified
Professional Exam Provider" (the "Logo") according to the Microsoft guidelines
attached hereto as Appendix K of the Statement of Work or as may be provided by
MCCL from time to time. ATA shall correct any specified misuses of Microsoft's
trademarks or the Logo. This Agreement does not grant ATA any right, title,
interest, or license in or to any of Microsoft's names, word marks, logos,
logotypes, trade dress, designs, or other trademarks. ATA is permitted, however,
to make descriptive references to Microsoft's non-stylized word marks (but may
not use Microsoft's logos other than the Logo authorized herein, or any
logotypes, trade dress, or designs) in product packaging, documentation,
advertising, and marketing materials, including worldwide web pages, according
to the Trademark Guidelines attached hereto as Appendix K of the Statement of
Work. ATA shall correct any specified misuses of Microsoft's trademarks. Nothing
herein shall restrict Microsoft's legal or equitable rights to protect its
Microsoft trademarks against infringement, dilution, or other misuse.
Notwithstanding the above, in the event that the Logo is discontinued and are
not created, ATA will not be permitted to use any of Microsoft's trademark
and/or Logo.
4.3 ATA Trademarks and Logos. Subject to ATA's review and prior written
approval, MCCL may use ATA trademarks and logos in connection with the MOE
Polytechnic Program. This Agreement does not grant MCCL any right, title,
interest, or license in or to any of ATA's names, word marks, logos, logotypes,
trade dress, designs, or other trademarks. MCCL shall correct any specified
misuses of ATA's trademarks. Nothing herein shall restrict ATA's legal or
equitable rights to protect its ATA trademarks against infringement, dilution,
or other misuse. Notwithstanding the above, in the event that any of ATA's logos
are discontinued and are not created, MCCL will not be permitted to use any of
ATA's trademark and/or logos.
4.4 Exam Delivery System. ATA is the owner of the exam delivery system,
including exam driver and engine software, processes and procedures and other
elements utilized in the performance of its obligations under this Agreement and
the Exhibits attached hereto ("Exam Delivery System"). MCCL is expected to
provide feedback to ATA to enable ATA to perform ATA's obligations, which ATA
may include in the Exam Delivery System, and MCCL will own such feedback.
Nothing in this Section 4.4 shall alter the provisions of Section 5 below.
5. Confidentiality.
5.1 The parties each expressly undertake to retain in confidence all
Confidential Information disclosed by one party to the other hereunder pursuant
to the terms and conditions set forth in that certain Non-Disclosure Agreement
which is attached hereto as Appendix J of the Statement of Work and incorporated
herein by this reference. The specific terms of this Agreement (including all
Exhibits attached hereto) shall be deemed to be Confidential Information
pursuant to the NDA.
6. Warranties.
6.1 Mutual Warranties. Each party hereby represents and warrants as
follows:
(a) Corporate Power. Such party is duly organized and validly existing
under the laws of the state of its incorporation and has full corporate power
and authority to enter into this Agreement and to carry out the provisions
hereof.
Page 3 of 13
(b) Due Authorization. Such party is duly authorized to execute and
deliver this Agreement and to perform its obligations hereunder.
(c) Binding Agreement. This Agreement is a legal and valid obligation
binding upon it and enforceable with its terms. The execution, delivery and
performance of this Agreement by such party does not conflict with any
agreement, instrument or understanding, oral or written, to which it is a party
or by which it may be bound, nor violate any law or regulation of any court,
governmental body or administrative or other agency having jurisdiction over it.
6.2 ATA Warranties. ATA represents and warrants the following to MCCL:
(a) ATA has been authorized by the Ministry of Education to launch
this MOE Polytechnic Program in China.
(b) The obligations to be performed by ATA to MCCL under this
Agreement and the Exhibits attached hereto do not infringe any copyright, trade
secret, trademark, or other proprietary right held by any third party;
(c) The obligations to be performed by ATA shall be performed in
compliance with all applicable laws and regulations of any relevant
jurisdiction;
(d) The obligations to be performed by ATA shall be performed by ATA
in a professional manner and shall be of a high grade, nature, and quality and
in conformity with the description set forth in the Statement of Work; and
(e) ATA's employees and agents shall at all times, while on MCCL
property or while performing its obligations, comply with all applicable local
and State laws, including specifically all laws prohibiting harassment of any
kind in the workplace. ATA assumes all responsibility for providing to its
employees, agents and subcontractors any training that may be required to insure
compliance with such laws.
6.3 MCCL Warranties. MCCL represents and warrants the following to ATA:
(a) MCCL has all rights necessary to perform its obligations under
this Agreement;
(b) MCCL's employees shall at all times, while on ATA's property,
comply with all applicable local and State laws, including specifically all laws
prohibiting harassment of any kind in the workplace. MCCL assumes all
responsibility for providing its employees training that may be required to
insure compliance with such laws.
EXCEPT AS EXPRESSLY SET FORTH IN THIS SECTION 6, NEITHER PARTY MAKES ANY
WARRANTIES WITH RESPECT TO ITS PRODUCTS, SERVICES OR PERFORMANCE HEREUNDER,
EXPRESS, STATUTORY, OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
7 Limitation of Liability.
NEITHER PARTY SHALL BE LIABLE TO THE OTHER PARTY FOR ANY CONSEQUENTIAL,
INCIDENTAL, PUNITIVE, OR SPECIAL DAMAGES (INCLUDING, WITHOUT LIMITATION, LOST
PROFITS) INCURRED BY THAT PARTY AS A RESULT OF ANY BREACH OF THIS AGREEMENT NOR
SHALL THE AGGREGATE LIABILITY OF EITHER PARTY IN ANY EVENT EXCEED TEN MILLION
DOLLARS ($10,000,000.00).
Page 4 of 13
8. Indemnity.
8.1 Indemnity by ATA. ATA agrees to indemnify and hold MCCL and its
successors, officers, directors and employees harmless from any and all third
party claims, including: actions, causes of action, claims, demands, costs,
liabilities, expenses and damages (including reasonable attorney's fees and
expenses incurred in connection therewith or with successfully establishing the
right to indemnification hereunder) to the extent arising out of:
(a) The breach of any ATA warranty set forth in Section 6 above or
default by ATA in the performance of any obligation to be fulfilled by ATA under
this Agreement, or any claim that if true would constitute a breach of ATA's
warranties set forth in Section 6 above; and
(b) ATA's performance of its obligations.
8.2 Indemnity by MCCL. MCCL agrees to indemnify and hold ATA, and its
successors, officers, directors and employees harmless from any and all third
party claims, including: actions, causes of action, claims, demands, costs,
liabilities, expenses and damages (including reasonable attorney's fees and
expenses incurred in connection therewith or with successfully establishing the
right to indemnification hereunder) to the extent arising out of:
(a) The breach of any MCCL warranty set forth in Section 6 above or
default by MCCL in the performance of any obligation to be fulfilled by MCCL
under this Agreement, or any claim that if true would constitute a breach of
MCCL's warranties set forth in Section 6 above;
(b) MCCL's performance of its obligations under this Agreement;
(c) Provided, however, that this indemnity expressly excludes any
claim set forth in this Section 8.2 above or any loss, cost, damage or expense,
to the extent arising out of the negligent or willful acts or omissions of: ATA,
its employees, agents, representatives, or any third party outside the control
or direction of ATA. For purposes of this Section 8.2(c), all testing centers,
ATA's employees, subcontractors and independent contractors are deemed to be
under the direction or control of ATA.
(d) Regardless of the basis of indemnification, under no circumstance
shall MCCL's obligation to indemnify exceed the Licensing Fees received by MCCL
for the trailing 12 months in which the basis for indemnification arises out or
is connected to.
8.3 Notice of Claim. If any action shall be brought against a party (the
"Indemnified Party") in respect to which indemnity may be sought pursuant to the
provisions of Sections 1.4, 1.5 and 8, the Indemnified Party shall promptly
notify the other party (the "Indemnifying Party") in writing, specifying the
nature of the action and the total monetary amount sought or other such relief
as is sought therein. The Indemnifying Party may, at its option, assume the
defense of the action, in which event the Indemnified Party will cooperate fully
in such defense and may participate in such defense with counsel of its own
choice, provided that the Indemnified Party will be responsible for all expenses
relating to such separate counsel. If the Indemnifying Party assumes the defense
of the action, its obligation will be limited to paying the attorneys' fees,
costs and expenses associated with such defense (except as otherwise expressly
provided herein) and holding harmless the Indemnified Party from and against any
judgment paid on account of such action or monetary settlement the Indemnifying
Party has made (with the Indemnified Party's approval, not to be unreasonably
withheld) or approved. No settlement may be made by either party without the
other party's prior approval, such approval not to be unreasonably withheld.
ATA AND MCCL EXPRESSLY AGREE THAT THIS SECTION ENTITLED INDEMNIFICATION HAS BEEN
READ AND REVIEWED AND HAS BEEN THE SUBJECT OF NEGOTIATION BETWEEN THE PARTIES,
AND THAT EACH AGREES TO BE BOUND BY THE TERMS THEREOF.
Page 5 of 13
9. Insurance
9.1 Coverage. Prior to the commencement of this Agreement and throughout
the entire term thereof and to the extent available in China, ATA shall maintain
the following insurance. Such insurance shall be in a form and with insurers
reasonably acceptable to MCCL, and shall comply with the following minimum
requirements:
(a) Comprehensive General Liability. ATA shall obtain and maintain a
policy of "general", "public", or "commercial" liability insurance with policy
limits of not less than $1,000,000 each occurrence for bodily injury and
$1,000,000 each occurrence for damage to property, or, alternatively, $1,000,000
combined single limit each occurrence for bodily injury and property damage
combined. The policy shall be the "occurrence" form, including coverage for
premises and operations, contractual liability (including specifically liability
assumed herein), broad form property damage, and products and completed
operations. ATA shall name MCCL, its subsidiaries, and their respective
directors, officers and employees as additional insureds under such policy to
the extent of contractual liability assumed by ATA under this Agreement. The
coverage afforded to the additional insureds shall be primary and
non-contributory to insurance maintained by MCCL, and shall contain a
severability of interests provision in favor of the additional insureds.
(b) Automobile Liability. If licensed vehicles will be used in
connection with the performance of the work hereunder, and at all times when
such vehicles are operated on the premises of MCCL, ATA shall maintain
automobile liability insurance covering all owned, rented, and non-owned
vehicles operated by ATA with policy limits of not less than $1,000,000 combined
single limit for bodily injury and property damage combined, or, if limits are
obtained on a per person and per accident basis, not less than $1,000,000 per
person and per accident for bodily injury, and $500,000 per accident for
property damage.
(c) Workers' Compensation. ATA shall at all times and to the fullest
extent comply with all applicable workers' compensation, occupational disease,
and occupational health and safety laws, statutes, and regulations. Compliance
and coverage shall extend to all employees of ATA suffering bodily injury
(including death) by accident or disease, which arises out of or in connection
with the performance of this Agreement. Satisfaction of these requirements shall
include, but shall not be limited to:
(i) full participation in any required governmental occupational
injury and/or disease insurance program, to the extent participation in such
program is mandatory in any jurisdiction, and
(ii) purchase of workers' compensation and occupational disease
insurance providing benefits to employees in full compliance with all applicable
laws, statutes, and regulations (but only to the extent such coverage is not
provided under a mandatory government program as in Section 9.1(c)(i) above),
and/or
(iii) maintenance of a legally permitted and governmentally
approved program of self insurance for workers' compensation and occupational
disease.
If ATA fails to effect and maintain a program of compliance with applicable
workers' compensation and occupational disease laws, statutes, and regulations,
and MCCL incurs fines or is required by law to provide benefits or to obtain
coverage for such employees, ATA shall indemnify MCCL for such fines, payment of
benefits to employees or ATA or their heirs or legal representatives, and/or the
cost of effecting coverage on behalf of such employees. Any amount owed to MCCL
by ATA pursuant to this indemnity may be deducted from any payments owed by MCCL
to ATA for performance of this Agreement.
(d) Employer's Liability. ATA, in addition to complying with the
provisions of section (c) above, shall maintain coverage for employer's
liability with a policy limit of not less than $1,000,000 per accident. In
states where commercial insurance of workers' compensation is not permitted,
Page 6 of 13
this requirement may be fulfilled through addition of the "Employers Stop Gap
Liability" endorsement to the comprehensive general liability policy required in
(a) above.
(e) Errors & Omissions Liability Insurance. Policy limits of not less
than One Million Dollars ($1,000,000.00) each claim. Throughout the term of this
Agreement, the Errors & Omissions Liability Insurance retroactive coverage date
will be no later than the Effective Date of this Agreement ("Effective Date" is
defined in Section 10.1 below). Upon expiration or termination of this
Agreement, ATA will either (i) continue to maintain such coverage for a period
of at least one (1) year following the termination of this Agreement; (ii)
maintain for at least one (1) year full replacement coverage that also complies
with the requirements of this Section 9.1(e); or (iii) maintain, upon terms
consistent with all other pertinent terms of the policy then in force, an
extended reporting period ending not less than one year following termination of
this Agreement ("Extended Reporting Period"), and providing that claims first
made and reported to the insurance company within said Extended Reporting Period
will be deemed to have been made during the policy period.
9.2 General Requirements Applicable to All Above Coverages.
(a) All deductibles and premiums associated with the coverages
described in the above Section 9.1 shall be the responsibility of ATA.
(b) If, in the reasonable opinion of MCCL, the amount of liability
coverage is not adequate by reason of inflationary pressures or experience or
the nature and content of ATA's activities, ATA shall increase the amount of
insurance coverage as reasonably required by MCCL.
(c) The use of umbrella or excess liability insurance to achieve the
above required liability limits shall be permitted by MCCL, provided that such
umbrella or excess insurance results in the same type and amounts of coverage as
required under the required individual policies identified above.
(d) It is the intent of the parties that the above insurance
requirements will assure adequate resources for victim compensation and will
serve to minimize MCCL's involvement and liability arising out of performance of
this Agreement by ATA. Where any subcontractor is retained by ATA in the
performance of this Agreement, ATA shall ensure that ATA's Errors and Omissions
Liability Insurance and Comprehensive General Liability coverage hereunder shall
respond to a claim of liability for the negligence, errors, acts or omissions
(as applicable under the terms of the pertinent policy) of its subcontractors to
the extent that such coverage is commercially available.
9.3 Certificates of Insurance. Prior to the commencement of this Agreement,
ATA (and any subcontractor thereof where ATA is not assuming its insurance
obligations) shall provide to MCCL certificates of insurance evidencing full
compliance with the insurance requirements contained herein. Such certificates
shall be kept current throughout the entire term of this Agreement, and shall
provide for at least thirty (30) days' advance notice to MCCL if the coverage is
to be canceled or materially altered so as not to comply with the foregoing
requirements.
Where such insurance is to waive rights of subrogation or include MCCL as
an additional insured the certificate shall expressly reflect in writing the
insurers' acceptance of such requirements.
Failure by ATA to furnish certificates of insurance or failure by MCCL to
request same shall not constitute a waiver by MCCL of the insurance requirements
set forth herein. In the event of liability or expense incurred by MCCL as a
result of such failure by ATA, ATA hereby agrees to indemnify MCCL for all
liability and expense (including reasonable attorney's fees and expenses
associated with establishing the right to indemnity) incurred by MCCL as a
result of such failure by ATA.
10. Effectiveness, Term and Termination.
10.1 Effectiveness. This Agreement shall not become effective unless and
until (i) MCCL and ATA have signed this Agreement and stamp their corporate
seals, respectively; (ii) MCCL and ATA
Page 7 of 13
has signed the Statement of Work and the Appendices attached thereto and stamp
their corporate seals thereto; and (iii) ATA has signed with ATA Service
Providers (defined in the Statement of Work) agreements that truly and
accurately reflect ATA's obligations under this and the Statement of Work
("Effective Date"). ATA hereby agrees that it shall submit to MCCL an original
copy of the agreements to be executed by ATA and ATA Service Providers, and MCCL
reserves the right to audit such agreements.
10.2 Term. This Agreement shall commence as of the Effective Date. The
Agreement shall terminate upon the earlier of (i) June 30, 2004 or (ii) as
terminated under this Section 10. Notwithstanding anything contained herein,
this Agreement shall be automatically extended for an additional period of one
(1) year period unless either party provides written notice to the other party
of its intent to terminate the Agreement without cause pursuant to Section 10.5.
Termination of this Agreement will not affect (i) Licensing Fees for training
and/or exams furnished prior to termination or as mutually agreed after
termination; or (ii) MCCL's and ATA's compliance to the terms and conditions of
this Agreement in connection with ATA's obligations actually furnished. On or
before April 1, 2004 or upon any earlier termination notice, MCCL may notify ATA
of its intent to negotiate a further agreement between ATA and MCCL.
10.3 Default. This Agreement shall terminate at the sole discretion of the
non-defaulting party, upon a default under Sections 3 or 10.3(a)(2) below. MCCL
may terminate this Agreement upon a default under Section 10.3(a) (1).
(a) Each of the following is a Default or an event of default:
(1) The acquisition of a substantial part or all of ATA's assets
by a third-party or its merger with another company;
(2) The occurrence of any of the following (each an "Insolvency
Event of Default"): (i) any party admits in writing its inability to pay its
debts generally or makes a general assignment for the benefit of its creditors;
(ii) a proceeding is instituted, voluntarily or otherwise, by or against any
party seeking to adjudicate it a bankrupt or insolvent, or seeking
reorganization, arrangement, adjustment or composition of it or its debt, which
is not dismissed within thirty (30) days; (iii) any affirmative act of
insolvency by any party or the filing by or against any party of any petition or
action under any bankruptcy, reorganization, insolvency arrangement,
liquidation, dissolution or moratorium law, or any other law or laws for the
relief of, or relating to, debtors; (iv) a proceeding is initiated against any
party seeking to appoint a receiver, trustee or other similar official for it or
for any substantial part of its property; (v) a party ceases to pay its debts as
they become due; (vi) any party becomes Insolvent, as defined in Section 10.3(c)
herein; or (vii) the subjection of a material part of any party's property to
any levy, seizure, assignment or sale for or by any creditor, third party or
governmental agency.
(b) Effect of Termination and Survival
(1) Compensation. Notwithstanding termination of this Agreement,
each party shall return any Confidential Information of the other party within
ten (10) days from the date of such termination. In addition, ATA shall return
all data, solely as it is derived from MCCL facilities and ATA's obligations
performed under this Agreement, to MCCL within ten (10) days from the date of
such termination in a standard computerized management maintenance system
format.
(2) Transition. In circumstances as expressly provided by this
Agreement, or in the event of any complete or partial cessation of ATA's
obligations under this Agreement or any of ATA's obligations provided under this
Agreement (any or all of the foregoing referred to as a "COS"), in each instance
without regard to cause or reasons for such COS, including expiration or
termination of this Agreement, but expressly provided that MCCL is not in breach
or default of any of its confidentiality obligations hereunder, ATA and MCCL
agree to have an orderly transfer of ATA's obligations to another performer as
MCCL may designate over a period not to exceed thirty (30) calendar days
following other applicable notice periods and at charges equal to 100% of the
charges then applicable hereunder. In particular, but subject to the foregoing,
ATA agrees that:
Page 8 of 13
(A) The customer database and records of customer
transactions (including applications, orders and record of fulfillment) and
other pertinent customer or exam records, including electronic records, but
excluding ATA works, in ATA's possession or reasonable control and necessary to
answer MCCL customer inquiries and to provide support previously provided by the
ATA, shall be transferred expeditiously and without charge, except out-of-pocket
costs and for any programming required, to MCCL or its new designated service
performer;
(B) Promptly after it is apparent to either party that there
will be a COS, the parties shall meet to schedule and organize the shift of
ATA's obligations in order that it shall be seamless to the fulltime students
under the MOE Polytechnic Program and occur without undue expense or
inconvenience to either party; MCCL shall pay ATA for additional services
provided by ATA;
(C) Provided MCCL is not in breach or default of its
confidentiality obligations hereunder, ATA acknowledges its duties under this
subsection are independent of any other disputes or claims it may have against
MCCL, and that it shall not utilize its practical relationship and control over
prior performance of its obligations or records to achieve any advantage or
leverage in relation to such other disputes or claims; and
(D) The duties of the parties under this transition process
shall be specifically enforceable by court order because there would be
irreparable damage to MCCL, in the case of any interruption of ATA's
obligations.
(c) Insolvent. For purposes of this Agreement, Insolvent shall mean a
financial condition such as to make the sum of a party's debts greater than all
of the party's assets, at fair valuation; or, when a party has incurred debts
beyond that party's ability to pay such debts as they mature; or, when a party
is engaged in a business or transaction for which the party has unreasonably
small capital.
(d) Effect of Default. In the event of a default, the parties shall
have all rights and remedies provided in this Agreement or otherwise available
under law as limited by this Agreement.
10.4 Termination With Cause. Either party may terminate the Agreement
immediately upon written notice to the other party specifying the breach and
intent to terminate for breach of Sections 5, or 11.2. MCCL may terminate the
Agreement upon written notice to ATA for breach of any payment obligation in the
event ATA fails to cure such breach to MCCL's satisfaction within fifteen (15)
days of written notice of such breach specifying the breach and intent to
terminate. Otherwise, either party may terminate upon written notice in the
event the other party fails to cure any breach of this Agreement within thirty
(30) days of written notice specifying the breach and intent to terminate of
such breach.
10.5 Termination Without Cause. Either party may terminate the Agreement
without cause or the occurrence of a default upon ninety (90) days' written
notice to the other party.
10.6 Survival. In the event of termination of this Agreement for any
reason, any unsatisfied payment obligations of ATA and Sections 1.2, 4, 5, 6, 7,
8, 9, 10.3(b), 11, 12.3 and 12.5 shall survive. With respect to tax matters, the
terms and conditions of Section 12.4 shall survive termination until the
expiration of any applicable statute of limitations or extension thereof.
10.7 Maintenance of Records and Reports Upon Termination. If this Agreement
is terminated for any reason, ATA will store all records and reports for at
least one hundred and eighty (180) days after the termination date, and make
such records and reports available to MCCL at no additional cost.
11. Audit Rights.
11.1 (a) ATA shall keep all usual and proper records related to its
obligations described in this Agreement and the Statement of Work. MCCL reserves
the right, upon five (5) business days notice, to audit ATA's records and
consult with ATA's accountants for the purpose of verifying ATA's
Page 9 of 13
compliance with the terms of this Agreement and invoices and/or statements
issued by ATA, during the term of this Agreement and for a period of two (2)
years thereafter, provided that any such audits shall be conducted during normal
business hours in such a manner as to not unreasonably interfere with the normal
business operations of ATA.
(b) Any such audit shall be paid for by MCCL unless material
discrepancies are disclosed. "Material" shall mean a discrepancy of two percent
(2%) or higher between amounts billed to MCCL and ATA records. If any material
discrepancy is disclosed ATA agrees to pay MCCL for the reasonable costs
associated with the audit. ATA agrees to promptly correct any deficiencies
detected in the audit and shall promptly refund any overpayments disclosed by
such an audit, or MCCL may, at its election, set off any such overpayment
against any money subsequently due by MCCL to ATA. In no event shall audits be
made more frequently than twice annually unless the immediately preceding audit
disclosed a material discrepancy. MCCL recognizes and agrees that information
contained therein is confidential and will use it only in furtherance of
disposition of the audit in question.
11.2 Confidentiality. Notwithstanding the foregoing, ATA may edit its books
and records to protect confidential information of ATA that is unrelated to the
subject of a MCCL audit, or to protect confidential information of the customers
of ATA.
12. Miscellaneous.
12.1 Independent Contractors. ATA is an independent contractor for MCCL,
and nothing in this Agreement shall be construed as creating an
employer-employee relationship, a partnership, or a joint venture between the
parties.
12.2 Assignment. ATA may not assign this Agreement, or any rights or
obligations hereunder, whether by operation of contract, law or otherwise,
except with the express written consent of MCCL, and any attempted assignment by
ATA in violation of this Section shall be void. For purposes of this Agreement,
an "assignment" by ATA under this Section shall be deemed to include, without
limitation, each of the following: (a) a change in beneficial ownership of ATA
of greater than twenty percent (20%) (whether in a single transaction or series
of transactions) if ATA is a partnership, trust, limited liability company or
other like entity; (b) a merger of ATA with another party, whether or not ATA is
the surviving entity; (c) the acquisition of more than twenty percent (20%) of
any class of ATA's voting stock (or any class of non-voting security convertible
into voting stock) by another party (whether in a single transaction or series
of transactions); and (d) the sale or other transfer of more than fifty percent
(50%) of ATA's assets (whether in a single transaction or series of
transactions). In the event of such assignment or attempted assignment by ATA,
MCCL shall have the right to immediately terminate this Agreement.
This Agreement, and any rights or obligations hereunder, may not be
assigned or transferred by ATA without MCCL's prior written approval. For
purposes of this Agreement, a merger, consolidation, or other corporate
reorganization or a transfer or sale of 51% of the voting shares of ATA's stock,
or of all or substantially all of its assets shall be deemed to be an
assignment. This is a contract for personal services and MCCL relies upon the
qualifications, reputation and expertise of ATA to perform all obligations
hereunder. In particular, MCCL relies on ATA's history of performance over
thirty (30) years of operation.
12.3. Notices.
All notices and requests in connection with this Agreement shall be deemed
given as of the day they are received either by messenger, delivery service, or
in the United States of America mails, postage prepaid, certified or registered,
return receipt requested, and addressed as follows:
Page 10 of 13
NOTICES TO ATA: NOTICES TO MCCL:
- -------------- ---------------
ATA Microsoft (China) Co., Ltd.
Bei Hai Wan Tai Building Room 707 ADDRESS
No.6 North Chao Yang Men Street, 6F, Beijing Sigma Center
Beijing P.R.China 100027 No. 49, Zhichun Road, Haidian District
Attn.: Mr. Wang Lin Beijing 100080, P.R.C
Attn: ATA Account Manager
Telephone: 86-10-85282060
Fax: 86-10-85282050 Copy to: Law & Corporate Affairs
Copy to: General Counsel Telephone: (86 10) 6261-7711
ATA Fax: (86 10) 6255-5402
Bei Hai Wan Tai Building Room 707
No.6 North Chao Yang Men Street, Copy to: Jay Henningsen
Beijing P.R.China 100027 Group Product Manager
Microsoft Corporation
Telephone: 86-10-85282060 1 Microsoft Way
Fax: 86-10-85282060 Redmond, WA 98052
Telephone: (425) 706-4441
Fax: (425) 706-4329
or to such other address as the party to receive the notice or request so
designates by written notice to the other.
12.4 Taxes.
(a) The parties acknowledge that ATA may launch the MOE Polytechnic
Programs in Hong Kong S.A.R., Macao S.A.R. and Taiwan, with prior written
approval from MCCL. ATA shall be responsible for any income, franchise, excise,
sales, use, gross receipts, value added, goods and services, property or similar
tax imposed by any foreign, federal, state, provincial, municipal, local and
other taxing authority which is required to be paid by ATA as a result of the
transactions contemplated under this agreement. MCCL is not liable for any
taxes, duties, levies, fees, excises or tariffs or any penalties, interest or
any additions thereon, incurred in connection with or related to the sale of
goods and services under this Agreement. Such taxes shall be the financial
responsibility of ATA, including penalties, interest and other additions
thereon, and ATA agrees to indemnify, defend and hold MCCL harmless from any
taxes or claims, causes of action, costs (including without limitation,
reasonable attorneys' fees), penalties, interest charges and other liabilities
of any nature whatsoever related to such taxes.
(b) All amounts to be earned by ATA herein are inclusive of all taxes
imposed by all foreign, federal, state, provincial, municipal, local and other
taxing authorities, including income, franchise, excise, sales, use, gross
receipts, value added, goods and services, property or similar tax, now or
hereafter imposed on ATA. Such taxes shall be the responsibility of ATA and may
not be passed on to MCCL.
12.5 Governing Law and Dispute Resolution. This Agreement shall be governed
by the laws of the People's Republic of China (for the purpose of this Section
12.5, the laws of the People's Republic of China does not include Hong Kong
S.A.R., Macao S.A.R. and Taiwan). The parties hereby agree to settle disputes
arising out of this Agreement through friendly negotiations. If any dispute
cannot be settled through friendly negotiations within 60 days after either
party sends out a notice for negotiation, then either party may submit the
dispute to arbitration at the China International Economic and Trade Arbitration
Commission in Beijing (CIETAC) in accordance with its rules in effect from time
to time. The arbitration award shall be final and binding upon both parties. In
any action or suit to enforce any right or remedy under this Agreement or to
interpret any provision of this Agreement, the prevailing party shall be
entitled to recover its costs, including reasonable attorneys' fees. This choice
of jurisdiction does not prevent either party from seeking injunctive relief
with respect to a violation of intellectual property rights or confidentiality
obligations in any appropriate jurisdiction.
12.6 Subcontracting. ATA shall not have the right to subcontract any
portion of its obligations under this Agreement and the Statement of Work to
third parties who are significant competitors in relation to MCCL's core
packaged software development and sales business without the
Page 11 of 13
prior written approval of MCCL. MCCL shall have the right to subcontract any of
its obligations under this Agreement without the prior written approval of ATA.
If ATA is unsure whether an entity is a significant competitor, ATA agrees that
it will contact MCCL for its determination. This Section 12.6 shall not limit
ATA's ability to enter into Agreements with prospective testing centers.
12.7 Construction. If for any reason a court of competent jurisdiction
finds any provision of this Agreement, or portion thereof, to be unenforceable,
that provision of the Agreement shall be enforced to the maximum extent
permissible so as to effect the intent of the parties, and the remainder of this
Agreement shall continue in full force and effect. Failure by either party to
enforce any provision of this Agreement shall not be deemed a waiver of future
enforcement of that or any other provision. This Agreement has been negotiated
by the parties and their respective counsel and shall be interpreted fairly in
accordance with its terms and without any strict construction in favor of or
against either party.
12.8 Entire Agreement. This Agreement does not constitute an offer by MCCL
and it shall not be effective until the conditions specified in Section 10.1
hereinabove have been satisfied. This Agreement, all Exhibits hereto, and the
Non-Disclosure Agreement shall constitute the entire agreement between the ATA
and MCCL with respect to ATA's obligations and all other subject matter hereof
and merge all prior and contemporaneous communications. It shall not be modified
except by a written agreement dated subsequent to the Effective Date and signed
on behalf of ATA and MCCL by their respective duly authorized representatives.
12.9 Force Majeure. Neither party shall be responsible nor be deemed to be
in default under this Agreement for any delay or failure in performance of any
of its obligations to be performed hereunder resulting from causes beyond its
reasonable control and without its negligence (collectively referred to herein
as "Force Majeure"). Such causes shall include acts of God, strikes, lockouts,
other labor disturbances, riots, insurrections, civil disturbances, sabotage,
embargoes, blockades, acts of war, power failures, or communication line
failures.
Page 12 of 13
IN WITNESS WHEREOF, the parties have entered into this Agreement as of the
Effective Date written above.
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ---------------------------------------
Signature Signature
- ------------------------------------- ---------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ---------------------------------------
Title Title
- ------------------------------------- ---------------------------------------
Date Date
Page 13 of 13
Exhibit A
Exhibit A
Statement of Work
Page 1 of 30
Exhibit A
Table of Contents
Page 2 of 30
Exhibit A
Page 3 of 30
Exhibit A
This STATEMENT OF WORK ("SOW") is entered into on May 16, 2003, by and between
(CHINESE CHARACTERS) ATA Testing Authority, Beijing Co., Ltd. ("ATA") and
(CHINESE CHARACTERS) Microsoft (China) Co., Ltd. ("MCCL"), as Exhibit A to the
Master Service Agreement executed by and between MCCL and ATA on May 16, 2003
(THE "MASTER SERVICE AGREEMENT"), and shall be effective as of the Effective
Date as provided in the Master Service Agreement.
The purpose of this SOW is to further define the rights and obligations of ATA
and MCCL under the Ministry of Education ("MOE") Polytechnic Program (FYB011449)
in accordance with the principles as set forth in the Master Service Agreement.
Unless otherwise defined, all the defined terms under this SOW shall have the
same meaning as defined in the Master Service Agreement.
1.0 SCOPE OF THE MOE POLYTECHNIC PROGRAM
MCCL has been properly authorized and developed an education program based
on Microsoft Official Curriculum, the Microsoft Certified Professional
(MCP) Program and other Microsoft materials. MCCL will license these
programs to ATA in accordance with the Master Service Agreement and this
SOW, and ATA agrees to perform its obligations in accordance with the
Master Service Agreement and the Exhibits thereto.
In general, ATA agrees to perform the following obligations in accordance
with the Master Service Agreement and this SOW:
- Develop and maintain Candidate (defined in Section 3.0 hereinafter)
registration and record management
- Manage printing and distributing of printed Courseware Materials
(defined in Section 2.0 hereinafter)
- Register Candidates and provide appointment notification services for
associated exams
- Deliver exams in a secure, proctored environment
- Delivering exam results to Candidates, including beta exam rescores
- Collect Candidate data and remit to MCCL
- Collect exam results and remit to MCCL
- Provide program information and support to Candidates
- Provide exam certificates, MCP welcome kits and course completion
certifications to Candidates who complete courses and pass MCP exams
- Provide reporting to MCCL
- Payment and remittance of Licensing Fees to MCCL
- Recruit Customers (defined in Section 2.1 hereinafter)
- Develop and execute program marketing materials and Candidate
recruitment services
- Ensure the quality of its performance of the above obligations
Specifically, ATA shall perform each of the obligations as set forth in
this SOW, and each of the above obligations is deemed as "material" under
Section 3 of the Master Service Agreement.
Regardless whether the services will be performed by ATA itself or ATA will
hire any service providers to help to perform ATA's obligations in strict
accordance with the Master Service Agreement and this SOW ("ATA Service
Providers"), ATA shall be the party who is responsible for the performance
of such obligations to MCCL and be liable to MCCL for the failure of the
performance.
Unless otherwise specifically provided for in this SOW or in the Master
Service Agreement, for all of the obligations that ATA shall perform under
this SOW, MCCL is not obliged to make any payment to ATA.
2.0 SCOPE OF THE MANAGEMENT OF PRINTING AND DISTRIBUTING PRINTED COURSEWARE
MATERIALS
ATA shall manage the printing and distribution of the courseware materials,
including, without limitation to, printing of books, documents and
components, CDs and DVDs replication, and kitting and fulfillment services
in accordance with the Master Service Agreement and this SOW (such books,
documents and components, CDs and DVDs, kitting, and other materials to be
designated by MCCL from time to time are hereby collectively referred to as
"COURSEWARE MATERIALS".
Page 4 of 30
Exhibit A
ATA hereby represents and warrants that it has been authorized by the State
law and/or regulation to replicate CD and DVD, or ATA will find a service
provider to replicate such CD and DVD.
ATA will hire ATA Service Providers to (i) perform the printing work of
Courseware Materials, (ii) manage and operate a warehouse to control the
stock for the Courseware Materials that MCCL licenses ATA to print and
distribute in accordance with the Master Service Agreement and this SOW,
and (iii) to deliver such Courseware Materials to the Customers. ATA will
negotiate a market competitive price with ATA Service Providers. MCCL shall
have the right to review the negotiated price and refuse or accept the
negotiated price. ATA will renew any negotiations if necessary.
2.1 SERVICE LEVEL REQUIREMENTS
ATA shall contract with ATA Service Providers and meet the following
service level requirements for printed Courseware Materials:
- Print run turnaround from print ready file availability to
availability for Customer delivery within 15 business days.
("CUSTOMER" shall mean the Schools enrolled in the MOE
Polytechnic Programs and sign the Training Agreement with ATA in
the form and substance of Exhibit B to the Master Service
Agreement)
- Print overages exceeding 1% of the quantity ordered shall not be
accepted, and ATA shall be fully responsible for such overage.
- Only paper contained in the will be charged.
- Customer order fill rate - 99% within 72 hours.
- Delivery time to Customer - 10 business days.
- Published Customer service phone number
- Issue resolution within 48 hours (2 business days).
- Data feed - delivered Monday (Beijing time) for preceding week.
- Data errors resolved and resent within 24 hours.
2.2 MANAGEMENT SERVICES
To support ATA's efforts, MCCL will provide the following services to ATA
regarding ATA's management of printing and distribution of Courseware
Materials in support of the MOE Polytechnic Program. ATA shall communicate
the required actions to ATA Service Providers to efficiently implement the
production of Courseware Materials and delivery to Customers.
- Manufacturing specifications. MCCL shall provide to ATA a
manufacturing specification, which may change from time to time or by
title. This specification will include requirements for trim size,
paper type, cover stock, color specifications, and cartoning. ATA
shall urge ATA Service Providers to fulfill their obligations in
accordance with the practices and procedures that are equivalent to or
otherwise in accordance with the best industry practice and standards.
- Print run quantity management. This involves specification by MCCL of
the quantity of first run and reprint printings. ATA shall communicate
the quantities provided by MCCL to ATA Service Providers to enable a
maximum fifteen (15) business day turnaround from the time that print
ready files are available to the time that printed Courseware
Materials are available for shipment from the central warehouse.
- Inventory management. Inventory management shall consist of
maintaining inventory levels to meet shipment demands, ordering
destruction of obsolescent or over-produced Courseware Materials,
managing instructions for Courseware Materials' recalls or reworks as
appropriate, and title status designation as available for sale, on
hold, out of stock and others as appropriate. MCCL shall provide to
ATA instructions concerning inventory management. ATA shall
communicate all instructions from MCCL to ATA Service Providers to
ensure proper inventory maintenance. MCCL agrees to adjust royalties
due for obsolete Courseware Materials produced under the direction of
MCCL, except for
Page 5 of 30
Exhibit A
Courseware Materials with errors or omissions not caused by MCCL.
Should any of these inventory management instructions not be properly
communicated and executed by ATA Service Providers, ATA will bear full
costs associated with disposal of obsolescent or over-produced
Courseware Materials. Any damage to the Courseware Materials incurred
while being stored at ATA Service Providers' facility due to
negligence or improper storage will be at the expense of ATA or ATA
Service Providers rather than MCCL. ATA will provide MCCL with full
reporting of all inventory activities in accordance with Appendix L
attached hereto, ATA Revenue Reconciliation Reports.
- Distribution management. ATA and ATA Service Providers shall manage
and control the distribution network needed to deliver the Courseware
Materials as required in the Master Service Agreement and this SOW.
MCCL shall provide consulting services to ATA regarding distribution
policies and procedures. Such policies would include order fill rates,
delivery time metrics, customer service handling, call center metrics,
and the like.
ATA understands that the Courseware Materials are specifically for the
MOE Polytechnic Program and shall be distributed to Candidates only.
Accordingly, ATA hereby commits that it shall obey such obligations
and shall urge and supervise ATA Service Providers and Customers to
obey such obligations as well.
2.3 SUB-CONTRACTORS COSTS
ATA shall be solely responsible for the fees of ATA Service Providers and
nothing in this SOW shall authorize ATA to pledge the credit of MCCL or to
incur any commitment, liability or obligation upon MCCL.
3.0 SET-UP AND MAINTENANCE OF CALL CENTERS
ATA shall, at its own cost, provide all full-time students taking MCP
classes (and/or other classes to be designated by MCCL from time to time)
at Customers (such students are referred to as "CANDIDATES") with toll-free
telephone access (where available and not cost prohibitive) to a Microsoft
Regional Education Service Center ("RESC") for either registration or
inquiry purposes.
ATA may change the telephone/fax numbers with sixty (60) days written
notice to MCCL. ATA shall provide a forwarding number to Customers and
Candidates via (1) recorded message on the old number, lasting a minimum of
six (6) months from its termination; and (2) proactive notification on the
ATA Web site. In the event that local telephone service infrastructure
cannot support these requirements, ATA shall notify MCCL within fourteen
(14) days of this discovery.
All calls shall be answered with: "Microsoft Certified Professional
Program".
3.1 BASIC ACTIVITIES
ATA is responsible to fulfill its obligation to support Customers and
Candidates. ATA shall provide all necessary services to adequately fulfill
Candidates' needs as they relate to registration, training, testing and
issuing certificates. Specifically, these obligations shall include but are
not limited to the following:
- ATA shall receive incoming calls for registrations or inquiry.
- ATA shall provide online registration services.
- ATA shall provide registration services no less than from 7AM to 7PM
(Beijing time) Monday through Friday. ATA shall notify MCCL at least
thirty (30) days in advance of any planned RESC closing due to
holidays or any other reasons.
- ATA shall make outbound calls as required to fulfill ATA's
obligations. Outbound calls to Customers shall not be used for direct
marketing purposes.
Page 6 of 30
Exhibit A
- ATA shall receive and process incoming faxes as required to fulfill
ATA's obligations.
- ATA shall send outbound faxes as required to fulfill ATA's
obligations.
- The Fax Server shall have capability as required to fulfill ATA's
obligations.
- ATA shall appoint sufficient number of Customer Service
Representatives ("CSRs") to meet obligations, who shall have access to
the Web as required to fulfill ATA's obligations, and ATA shall submit
a list of the CSRs to MCCL in advance for approval.
- CSRs shall have e-mail as required to fulfill ATA's obligations.
- CSRs shall be subject to periodic call monitoring.
- CSRs shall continue to have access to Microsoft Candidate Manager (and
other required Microsoft systems and tools as they become available)
to fulfill ATA's responsibilities based upon a mutually agreed upon
technology phase-in plan.
- ATA shall manage and record all incoming complaints from Customers
and/or Candidates regarding the quality or delivery service of exams
and/or Courseware Materials. ATA shall work with ATA Service Providers
to resolve such issues and provide to the Customer with any
replacement materials within (5) five business days after receiving
the complaint.
3.2 KEY PERFORMANCE INDICATORS ("KPIS", REFER TO APPENDIX A, KPI REPORTING
REQUIREMENTS)
Progress towards the monthly-measured KPIs listed in this section shall be
reported on a weekly basis, unless another timeframe has been mutually
agreed.
ATA shall provide certification forecast reports to MCCL on a quarterly
basis (or more frequently as mutually agreed). This report will be broken
down by the major regions in which ATA provides a call center support
function.
- MCCL will work with ATA to convert the certification forecasts to exam
delivery forecasts, as requested by ATA.
- ATA will determine staffing needs to meet the call-to-registration
ratios. For example, the following ratios might be used: 1.2 calls per
forecasted delivery.
Unless otherwise provided for in this SOW, ATA must comply with the
defined metrics of the KPIs. If ATA does not meet the defined metrics
of the KPIs, MCCL reserves the right to impose agreed upon remedies as
set forth in the Master Service Agreement.
- PHONE SERVICE LEVEL (PSL) measures the percentage of calls answered
within ninety (90) seconds or less. The time begins at the later of:
(1) when the connection is made, (2) any prerecorded broadcast
messages have been played; or (3) the Candidate chooses to talk to a
call representative from a phone menu.
BENCHMARK: a minimum of 80% of inbound calls each month shall be
answered by a CSR within 90 seconds or less. This shall be reported
monthly by the RESC to MCCL. In the event this benchmark is not met,
ATA shall report reasons for such performance and actions taken to
correct, along with completion dates, to MCCL.
Page 7 of 30
Exhibit A
ATA shall report to MCCL monthly the total number of calls received at
each RESC. In addition, if the PSL is not met in accordance with the
benchmark requirements as set forth in the preceding paragraph, MCCL
may request that ATA report these metrics weekly until the metric has
been met for four (4) consecutive weeks.
These metrics will be reported to MCCL via a mutually agreed upon
format.
- PERCENTAGE OF CALLS HANDLED (PCH) measures the percentage of calls
received at the call center, which are handled, or answered, by a CSR.
(This metric is also referred to as the "abandonment rate".)
BENCHMARK: ATA shall handle a minimum of 96% of all calls received
each month (4% of calls are abandoned).
ATA shall report these metrics of each RESC monthly to MCCL. In the
event these benchmarks are not met, ATA shall report reasons for such
performance and actions taken to correct, along with completion dates,
to MCCL.
ATA shall report to MCCL monthly the total number of calls received by
each RESC and how many were not abandoned. In addition, if the metric
is not met, MCCL may request that ATA report these metrics weekly
until the metric has been met for four consecutive weeks.
- CHANNEL SUPPORT CALLS RECEIVED. ATA shall provide call center support
via phone, e-mail, Instant Messaging, or other communication
mechanisms for its testing sites. Support services include:
- Technical support
- Financial systems support
- Candidate escalation support
In the event that service level concerns arise from the channel, MCCL
reserves the right to request monthly regional PSL and PCH reports, as
well as time-to-resolve metrics, as a measurement of support service
quality.
3.3 MCP EXAM PROGRAM INFORMATION
The ATA CSR staff shall be familiar with the information contained within
Appendix B, MCP Customer Service Calls. Updated program information will be
relayed to ATA through, but not limited to, the following methods: e-mail,
ATA Brief (defined in Section 11.1 hereinbelow), contract, or SOW. ATA will
make program information easily accessible to each phone/order entry
representative and the phone/order center management staff.
Candidates requesting program information beyond that found in Appendix B,
MCP Customer Service Calls, should be referred to the Microsoft public Web
site or transferred to RESC as necessary.
3.4 STAFFING
ATA shall staff to handle the volume forecasts provided by MCCL, as
detailed in Section 3.2 ATA shall provide RESC staffing plans when
requested by MCCL. MCCL shall not request this report more than once per
quarter unless phone service metrics have been missed for one (1) full
month.
3.5 LANGUAGE CAPABILITIES
ATA's call center network shall support all languages that correspond with
each localized MCP exam at no additional cost to MCCL. Currently, these
languages include English and Chinese. In the event that future needs
change, MCCL will provide an ATA Brief (defined in Section 11.1 herein
below)to request that additional languages be supported.
Page 8 of 30
Exhibit A
3.6 RESC MOVES
ATA shall provide a minimum of thirty (30) day notice for any RESC move or
any system move that can reasonably be determined will affect the
scheduling or testing of Candidates. In the event of urgent circumstances
or an overriding business need, ATA will provide a report to MCCL
summarizing the move and its effect, if any, upon the MCP testing program.
3.7 CALL MONITORING
Where technology exists and local laws permit, MCCL shall be entitled to
remotely monitor incoming phone calls to RESCs, undetected by the CSR being
monitored. Additionally, where local laws do not prohibit recording, MCCL
shall be entitled to have calls being monitored and/or being recorded as
necessary.
3.8 TRAINING
ATA must employ a designated trainer that is knowledgeable of the MOE
Polytechnic Program and the MCP Program, as defined in Appendix B, MCP
Customer Service Calls. ATA shall publish training documents for MCCL
review, as requested by MCCL. MCCL may recommend specific areas of focus
for CSR training/retraining for the MCP program. MCCL shall provide
appropriate collateral training as necessary.
4.0 REGISTRATION AND SCHEDULING OF EXAMS
4.1 REGISTRATIONS
For each registration ATA shall ask the following questions and collect the
following information:
- Have you taken a Microsoft Certification test before?
- MCP ID number (required if Candidate has previously taken a test).
- First Name, Last Name, and middle initial (if applicable)
- Address, City, Province, Country, Postal Code
- Valid Country ID
- Internet e-mail address
- Company/organization name (if the Candidate does not have a
Company/organization name, the CSR may optionally enter "NO COMPANY")
- Telephone number (work and home)
- Fax number (Optional)
- For the first registration made by a new Candidate, ATA shall verify
and spell back all information listed above, and must verify that each
student is a full-time student at the Customers.
- For subsequent registrations, ATA shall verify all information listed
above.
4.2 CANDIDATE NOTIFICATION
ATA shall implement, wherever possible, a method for confirming to
Candidates their exam appointment time, which may include e-mail or other
selected method.
4.3 REGISTRATION AND EXAM AVAILABILITY
ATA shall take all necessary measures to ensure that instructors or
officials at Customers shall be required to register Candidates, and to
register within the time period that MCCL designates as the exam
registration period.
ATA shall not allow Customers to register Candidates for an exam before or
after the dates that MCCL designates as the exam registration period.
Page 9 of 30
Exhibit A
ATA shall not deliver an exam to any Candidate before or after the dates
that MCCL designates as the exam availability period without express
written consent from MCCL.
The start of the exam registration period typically precedes the exam
availability by 1-2 weeks for a beta exam, and by one (1) day for live
exams. (See also Section 12.2 for publishing information.)
The start or end date/time for registrations at a call center or test site
shall be the China local time, relative to the location of the call center
or test site.
The start or end date/time for exam delivery shall be the China local time
of the test site.
ATA shall make best efforts to disallow beta exam delivery at testing sites
who chronically fail to meet the 14-day Exam Data Sent KPI (see Section
8.6).
ATA shall make best efforts to disallow Candidates who have been banned
from the MCP program from registering for and taking exam. This includes
Candidates identified by ATA as having cheated, as well as identified as
being cheaters by other parties (see Section 10.7).
4.4 SAME-DAY REGISTRATIONS
ATA is permitted to disallow a Customer from requesting an exam delivery
date that coincides with the date of the registration request. ATA is
encouraged, however, to allow these "same-day registrations" whenever
possible, in order to further promote Customer satisfaction. In some
instances, ATA may accommodate same-day Candidates by scheduling them for a
time in "today's past," in order to maximize seating capacity. ATA must
take care when scheduling the test appointment to ensure there is
sufficient time to download the exam to the exam workstation, as this could
adversely impact the Exam Service Level KPI (see Section 9.1).
4.5 REGISTRATION REPORTING
ATA shall report to MCCL the total number of registrations taken and
registrations cancelled for Candidates, as well as the number of exams
delivered. These metrics shall be broken out by those received via a CSR at
a call center, those taken via the Internet, and those registrations taken
directly at the testing centers.
ATA shall report these metrics weekly via the KPI format outlined in
Appendix A, KPI Reporting Requirements.
MCCL may also require that registrations be broken out by exam type.
Beta exam registration reports shall be provided daily during the beta exam
period, beginning with the beta pre-registration period and ending at the
conclusion of the beta exam availability.
4.6 CANCELLATION AND RESCHEDULING POLICY
- CANDIDATE CANCELLATIONS. ATA will notify Candidates of cancellation
and rescheduling policies at the time of registration. This includes
registrations completed at call centers, testing sites, or via the ATA
Web site. ATA shall allow all Candidates the opportunity to cancel a
registration provided that the Candidate calls to cancel at least 24
hours prior to the scheduled appointment. The Candidate may also
reschedule an appointment with at least 24 hours notice prior to the
scheduled appointment. ATA may not set the cancellation deadline to be
greater than 24 hours prior to the appointment, as this impacts
customer satisfaction. ATA shall have discretion to be more
customer-friendly and make exceptions regarding cancellations and
reschedules requested with less than 24-hours advanced notice. If the
Candidate does not cancel or reschedule the registration prior to the
cancellation deadline, the Candidate shall be designated as a
"no-show."
Page 10 of 30
Exhibit A
If a voucher or promotional code is used to pay for an exam, the
Candidate may not reschedule an exam beyond the expiration date of the
voucher or promotion.
- ATA CANCELLATIONS. If ATA must cancel a Candidate registration for any
reason, ATA will notify the candidate at least one (1) business day
(24 hours) prior to scheduled exam. In the event that ATA cannot
contact the candidate prior to cancellation, ATA will make appropriate
compensation to candidate, such as a free exam at ATA's expense. ATA
will report all instances when a candidate's registration is cancelled
by ATA. This report should detail the reason(s) why the registration
was cancelled. ATA shall design this report, which is subject to
approval by MCCL. See Appendix E, Report Key, for additional
information on this report.
4.7 RETAKE POLICY
ATA shall enforce the MCP exam retake policy and not permit Candidates to
register or take an exam which violates this policy. This enforcement shall
be based on registration data from within ATA's system alone. When MS-Cert
integration is available, the enforcement shall be based on registration
data from within MS-Cert. MS-Cert is defined in Appendix D (ATA Exam
Results Import Format).
The policy for MCP exam retakes is:
If you do not pass an exam the first time, you may retake it at any
time. If you do not achieve a passing score a second time, you must
wait at least 14 days to retake the exam a third time. A 14-day
waiting period will be imposed for all subsequent exam retakes. If you
have passed an exam, you cannot take it again. Beta exams may be taken
only once.
Candidates are eligible for each course to take associated exam and
one re-take if initial exam ends in failure. Subsequent retakes must
to be attempted through commercial channel.
MCCL shall provide at least thirty (30) days notice to ATA in the event the
retake policy changes or a special case is necessary for a given exam.
5.0 QUALITY ASSURANCE AND CUSTOMER SATISFACTION
ATA will implement the following requirements for Customer surveys:
- Survey shall be an in-house survey. ATA is encouraged to supplement
with an independent research organization survey, but must keep their
survey data separate from the independent data.
- MCCL reserves the right to approve questions asked in the survey and
may request additional questions to be added to the survey. MCCL shall
make reasonable attempts to limit the number of survey questions. MCCL
shall not request survey question updates more than two (2) times per
year.
- Survey will be delivered to a statistically relevant, worldwide
population of MCP candidates within two (2) weeks after the Candidates
have taken an exam.
- Excessive annual survey costs may be managed by ATA by limiting survey
to key geographic areas, rotating regions surveyed, limiting sample
size and number of survey questions, and use of electronic survey
methods.
- Legal, technology, and other practical issues might limit the scope of
the survey in some regions. ATA will advise MCCL of any such
limitations.
- Survey results will be provided to MCCL on a monthly basis, or on a
mutually agreed timeframe. Results shall be provided to MCCL in a
summary report, with raw data made available upon request.
Page 11 of 30
Exhibit A
- Survey data will measure customer experience with testing at ATA,
starting with exam registration (though call centers, test sites, or
web registration), through exam delivery and post-exam requests (e.g.,
replacement score report requests)
MCCL also reserves the right to perform one or all of the following on a
reasonable basis at MCCL expense:
- MCCL Customer satisfaction survey
- Call monitoring
- Secret shopping of ATA's call center representatives and testing
centers
Based on the results of any or all of the above-mentioned survey or
methods, ATA may be asked to implement a mutually agreed upon performance
improvement plan to increase the quality of Candidate service in the call
center(s), test sites, or web.
All Customer surveys conducted by ATA shall be in accordance with local
laws. Consultation, if needed, may be provided by MCCL and MCCL service
providers.
6.0 INCIDENT TRACKING, RESOLUTION, AND REPORTING
To meet the needs of the MCP program in the event that an incident occurs
that causes or has the potential to cause a disruption in service to
Candidates, ATA shall notify MCCL as soon as an incident has occurred. In
addition ATA shall ensure the following are in place:
- Escalation path established for CSR, Testing Center channel, ATA
management, and MCCL contacts.
- Incident tracking system to capture information including, but not
limited to: date, Candidate name, issue type (categorized to
facilitate summary roll up), CSR or other point of initiation,
escalation path, resolution, date of closure.
- A MCCL Escalation Report shall be produced monthly. ATA shall design
this report, subject to MCCL's approval. The report should include
trend data as well as extraordinary incidents, in order to provide a
gauge of the health of the business.
- ATA is responsible to respond to all issues within one business day
and track the issue until resolved. ATA will make its best efforts to
resolve all issues within five (5) business days. In the event an
escalation must be forwarded to MCCL for further research, ATA will
continue to track status of the issue until resolved. ATA is also
responsible to compensate any Candidate affected by deficiencies of
ATA, at ATA's discretion.
7.0 DATA INTEGRITY
The following Sections 7 and 8 define requirements, processes, and metrics
regarding data and data feeds to MCCL. Additional requirements and
procedures are defined in, Appendix C, Missing Exam Information and Data
Feed Process. The sections below refer specifically to the current model
where ATA collects data from MCCL customers and remits data via a file
feed.
ATA is responsible to deliver all Candidate and exam data per the
processes, formats and KPIs outlined in the Appendix A, KPI Reporting
Requirements. In the event ATA is unable to correctly remit data to MCCL,
ATA is responsible to resolve any issues or to bring to MCCL's attention,
and continue to track until the issue is resolved. This includes all
records rejected by MS-Cert or failure to connect with a Microsoft server
to download data.
7.1 DATA INTEGRITY AUDIT
Page 12 of 30
Exhibit A
MCCL reserves the right to audit ATA's data as it pertains to Candidates no
more than twice per year at MCCL's expense. Audits will be scheduled at
least 10 business days in advance and occur during regular business hours.
MCCL may also request ATA to perform an audit of their data and publish the
results of that audit to MCCL. If such audit was not scheduled by ATA and
ATA incurs expenses related to the audit, MCCL should be responsible for
the expenses incurred, at ATA's request.
7.2 DEMOGRAPHIC CHANGES
ATA may update Candidates' demographic information as needed; however, for
security concerns, ATA is prohibited from updating Candidates' names.
Candidates must contact a Microsoft RESC for name changes. ATA shall update
the Candidate database for address changes received from the nightly MCCL
file feed within 48 hours of receipt. ATA shall report to MCCL monthly on
number of address changes completed within 48 hours.
7.3 MERGES
ATA is to complete all merge requests received from Candidates via RESCs,
within two (2) business days of receipt. ATA is to complete all merge
requests received from Candidates via MCCL, within one (1) business day of
receipt.
When a merge is requested, the duplicate Candidate information will be
merged into the original Candidate record. ATA will prevent duplicate
records from being created for Candidates who register via RESCs.
7.4 MISSING RESULTS
ATA will research and make best efforts to resolve any issue regarding an
exam record that is missing from either MCCL or ATA following first formal
notification by MCCL or its agents. This notification indicates that the
Candidate is aware that his or her record is incomplete and has escalated
to MCCL or MCCL agent. Quick resolution is critical. ATA shall make best
efforts to provide either resolution or a plan for resolution for missing
exam results within three (3) [business] days. In the event an escalation
must be forwarded to MCCL for further research, ATA shall continue to track
status of the issue until resolved. This issue should be considered an
escalation and rolled up with escalation reports listed under Section 6.0.
Missing exam results procedures are defined in Appendix C, Missing Exam
Information and Data Feed Process.
ATA MISSING EXAM RESULTS - ERROR FILE CORRECTION
ATA will research and make best efforts to resolve any issue regarding an
exam record that errors out from MCCL upon receipt of an exception report
from MCCL within five (5) days of the receipt of the exception report. In
the event that exception reports are not generated for a period of longer
than three (3) business days, ATA shall make best efforts to resolve these
issues within 5 days.
ATA MISSING EXAM RESULTS - ESCALATED ISSUES SPREADSHEET
ATA will research and make best efforts to resolve any issue regarding an
exam record that is missing from either MCCL or ATA following first formal
escalation by Microsoft Cert Tier 1 Support. This escalation indicates that
the Candidate is aware that his or her record is incomplete and has
escalated to MCCL or MCCL agent. Quick resolution is critical. ATA will
make best efforts to provide either resolution or a plan for resolution for
escalated exams within three (3) days.
ATA MONTHLY RESULT RESYNCHRONIZATION AND RECONCILIATION
ATA will perform an initial resynchronization and provide MCCL with the
results within five (5) days after the end of each calendar month. Upon
analysis by MCCL and return of the results of said analysis to ATA, ATA
will make best efforts to provide either resolution or a plan for
resolution for all outstanding results within five (5) days.
Page 13 of 30
Exhibit A
7.5 DATA CONFIDENTIALITY AND PRIVACY GUIDELINES
ATA agrees to comply with MCCL data confidentiality and privacy guidelines
that will be updated from time to time.
The following highlights the MCCL data protection and privacy principles.
ATA shall make best efforts to conform to these principles and shall
provide status reports to MCCL on their progress, to conforming to these
principles.
- NOTICE. Personal data shall be obtained only for one or more specified
and lawful purposes, and shall be collected only following adequate
notice of how the data will be used.
- CONSENT. Personal data shall not be used in any manner beyond the
specific purpose(s) for which the individual provided it, unless there
has been adequate disclosure of the additional purpose(s) and the
individual has consented.
- SCOPE & ACCURACY. Personal data collected shall be adequate, relevant,
and not excessive in relation to the purpose or purposes for which
they are collected. Personal data shall be kept accurate and up to
date, and shall not be kept for longer than is necessary for the
purpose or purposes for which it was collected.
- ACCESS. Individuals shall be given reasonable access to their personal
data and shall be able to correct or amend that data where it is
inaccurate (except as specified in Section 7.2 limits to correction).
- SECURITY. Appropriate technical and organizational measures shall be
taken against unauthorized or unlawful use of personal data and
against accidental loss or destruction of, or damage to, personal
data.
- REMEDY. Candidates shall be given a means of recourse in the event
that the stated privacy principles are not followed.
8.0 EXAM RESPONSE AND CERTIFICATION DATA FEEDS
This section provides requirements for Exam Response and Certification data
feeds to MCCL. ATA will comply with all requirements necessary to
successfully transmit data to MCCL. Additional requirements will be listed
in an ATA Brief (defined in Section 11.1 hereinafter) as necessary.
8.1 DATA FEED DEFINITIONS
See Appendix D, ATA Exam Results Import Formats, for definitions of
Candidate profile and exam results data to be returned to MCCL.
8.2 DATE AND TIME STAMP
ATA shall include a date and time stamp on all records. This date stamp
should reflect when a Candidate's demographic data has been updated as well
as when the Candidate actually completed an exam in the test centers time
zone. ATA shall make best efforts to ensure that their registration and
delivery system clocks are accurate and synchronized with their other
systems.
8.3 CONNECTIVITY
ATA will comply with MCCL connectivity requirements as outlined in Appendix
D, ATA Exam Results Import Formats.
8.4 SUPPORT
Page 14 of 30
Exhibit A
ATA will facilitate direct contact between customer's technical support and
maintenance team, and ATA's support staff. 24-hour support must be
available in the event of data problems. ATA must have its systems
configured to page ATA's support staff in the event of server or database
failure.
8.5 DATA FEED SCHEDULE
MCCL requires that at least one feed be sent to MCCL by 5:00 PM (PST) every
day. ATA may send more than one feed daily, but may do so only between the
hours of 5:00 AM (PST) and 5:00 PM (PST). ATA may not send feeds during
other times as this interferes with file processing within Microsoft
systems. In the event that a file is not received, ATA is required to
respond to MCCL within twelve (12) hours detailing reason why files were
not sent, action plan, and resolution date to correct.
8.6 KEY PERFORMANCE INDICATOR: EXAM DATA SENT
ATA shall deliver MCP exam results according to the following KPI
requirements:
- 90% of exam records within 72 hours (3 days) of exam delivery
- 100% of exam records within 120 hours (5 days) of exam delivery AND no
more than 500 late records
- 100% of exam records within 336 hours (14 days) of exam delivery
The timeframe metrics shall be measured from the time the Candidate
finishes the exam to the time that the exam record is first sent to MCCL
(i.e., the data is successfully downloaded to the Microsoft FTP site).
ATA shall report to MCCL this performance indicator on a monthly basis. In
the event this benchmark is not met, ATA shall report reasons for such
performance and actions taken to correct, along with completion dates.
8.7 DATA RETENTION
ATA will store and archive all exam data for a period of not less than five
(5) years from original exam delivery date and make data available to MCCL
upon request.
9.0 EXAM DELIVERY
ATA shall be responsible for delivering all of the MCP exams in a secure,
proctored environment and in a matter as required hereinbelow. This section
9 specifically provides the requirements for Exam Delivery at test centers.
9.1 KEY PERFORMANCE INDICATOR: EXAM SERVICE LEVEL
ATA is required to deliver 98% of all exams to properly enrolled Customers
on or before Candidate's scheduled exam. This KPI shall include same-day
registrations, except for those test appointments that are made for the
same day, but at a time prior of the registration transaction (see Sections
4.3 and 4.4). ATA shall report this to MCCL on a monthly basis. In the
event the metric is not met on a monthly basis, ATA shall report reasons
for such performance and actions taken to correct, along with completion
dates.
For those affected Candidates whose exams are not ready for delivery on
time, ATA shall offer compensation to these Candidates in order to maintain
customer satisfaction. This compensation will be most commonly in the form
of exam vouchers, for which ATA shall bear all costs, unless the reason for
the non-delivery is due to circumstances outside ATA's control as defined
as force majeure in Article 12.9 of the Master Service Agreement.
9.2 EXAM OFFERINGS
MCCL shall make available -- and ATA shall offer for delivery -- MCP
published exams offered through the MOE Polytechnic Program in the means to
be decided by MCCL in its sole discretion.
Page 15 of 30
Exhibit A
9.3 LOCALIZATION
MCCL requires ATA to support all localized exams published by MCCL.
Moreover, MCCL requires that ATA's systems support the delivery of exams in
double-byte languages. See also Section 12.2.
9.4 TIME DURATIONS AND EXTENSIONS
MCCL shall limit the exam session duration for live exams to four (4) hours
or less, and beta exams shall be limited to six (6) hours or less;
exceptions must be mutually agreed by ATA Testing and MCCL at least 30 days
in advance of the target publishing date. The exam session time includes
presentation of the NDA (defined in Section 9.5), demographic survey, exam
content, comment sections, and other sections MCCL might introduce in the
future.
9.5 EXAM NON-DISCLOSURE AGREEMENTS (NDA) AND PRIVACY STATEMENTS
Because Candidates must agree to a Non-disclosure Agreement (NDA) and
accept any Privacy Statement before being permitted to take an exam, ATA
must provide the following services:
- Proctors at the testing sites must be trained on the NDA policies for
those individuals who question it or refuse the agreement or
statement.
- The exam driver must have the ability to require Candidates to answer
Yes/No to the NDA and any privacy statement. If the Candidate chooses
No, it must verify the response, and if necessary, terminate the exam.
The responses (accept or decline NDA) must be captured.
- The exam driver must permit the NDA and privacy sections to be
excluded from the primary timed section of the exam.
- Candidates who refuse the NDA or Privacy Statement at the test site
(i.e., the exam is terminated) must not be scored as having failed the
exam. ATA will send data for this Candidate per Appendix D, ATA Exam
Results Import Formats.
10.0 SECURITY
10.1 EXAM SECURITY BREACH
ATA and MCCL shall work together to reduce piracy, cheating issues, and
incidents. ATA shall identify any potentially suspicious activities
regarding a Candidate taking an MCP exam. Each incident should be escalated
to MCCL and should be included in the Escalation Reporting described in
Section 6.0. Escalations may include questionable activity on the part of
an Microsoft Certified Professional or Microsoft Certified Trainer as well
as a Candidate taking a MCP exam. With respect to any incident of suspected
cheating, at MCCL's request, ATA shall provide MCCL with an official
document, executed by the proctor or other ATA support staff who observed
the cheating incident, setting out all relevant facts supporting the
conclusion that cheating occurred. MCCL reserves the right to take action
regarding Candidates, in addition to requiring ATA to take appropriate
action as mutually agreed to by ATA and MCCL.
10.2 TESTING CENTER SECURITY
Physical security provided by ATA at testing centers must address the
following issues:
- Live proctoring Testing Center Representative must be physically
on-site and ensure that:
- Positive identification is made before admitting Candidates using
valid government-issued photo identification (e.g., passport, driver's
license, state ID card, school ID card).
Page 16 of 30
Exhibit A
- Photograph of Candidate must be taken and included on his/her score
report and archived.
- Candidates read and sign a Test Center Rules and Regulations form that
describes test center and cheating policies. The text of this form
shall be mutually agreed to by MCCL and ATA.
- Candidates use only test-center-provided reusable wipe board and
marker. ATA test center shall provide the reusable wipe board and
marker to the Candidate prior to taking the exam and retrieve it upon
the Candidate's exit. No other foreign materials are allowed into the
testing area.
- Proper noise levels are maintained (including no talking between test
takers).
- Time limits are strictly enforced.
- The proctors and test site administrators at a test center may not be
current MCPs (see definition in the following paragraph) and may not
pursue taking new MCP exams. Instructors may not proctor their current
Candidates.
Proctors and test site administrators are considered to be a current
MCP if their credential is still valid (i.e, their exam(s) have not
retired); or if they are not current MCPs, they have failed any MCP
exam within the previous twelve (12) month period.
- Test Candidate must take the exam without aid from external reference
materials (e.g., books, crib sheets, Internet web sites, audio
devices, photo wristwatches and other espionage technologies) or from
any person. ATA test centers will make reasonable efforts to identify
external reference materials as given in the above example.
- Test stations must be visually separated from one another (privacy
panels may be used).
- Candidates should have no access to any exams before or after their
official testing session.
10.3 TEST CENTER COMPLIANCE
The reputation of the MCP Program is based in part on the security and
integrity of the exam taking experience. ATA shall make every effort to
effectively communicate Microsoft's policies and procedures to the
Customers and to ensure that all Customers are compliant with such policies
and procedures. In addition, ATA must define and maintain a quality control
and assurance program to ensure Customers adhere to Microsoft's policies
and procedures. The primary objective is to initiate investigation of
deviations from these policies and procedures within five (5) business days
after said deviation is discovered. In the event that a Customer is
suspected of deviations from Microsoft's policies and procedures, ATA shall
take any reasonable steps to remedy the situation in a reasonably short
period of time. In the event that ATA terminates a Customer's Training
Agreement in accordance with Exhibit B of the Master Service Agreement, ATA
will take reasonable steps to ensure that Candidate coverage is not
affected, by identifying alternative locations for test delivery. On a
monthly basis, ATA will submit summary reports to MCCL that include summary
of findings, root cause analysis, corrective action taken, and ongoing
measurement.
10.4 SECRET SHOPPING
ATA shall perform both random and targeted secret shopping of its testing
centers via video monitoring or other agreed means. Any video monitoring
tapes, CDs, etc will be archived for no less than five (5) years by ATA and
ATA shall make the same available for MCCL upon MCCL's request.
MCCL also reserves the right to secret shop ATA testing centers. If ATA or
MCCL discovers suspicious activity, ATA shall continue to investigate the
test center(s) in question, and/or take appropriate action, up to and
including de-authorization of MCCL testing or permanent site closure.
Page 17 of 30
Exhibit A
10.5 SITE CLOSURES DUE TO SECURITY INFRACTIONS
In the event that ATA shuts down a testing site for deviations from
ATA/MCCL standards, ATA shall ensure that such site is not allowed to
deliver tests until MCCL and the vendor(s) have determined the site is
eligible to begin delivering MCP exams again, at which time the site can
resume delivering MCP exams through any or all vendors. In addition, MCCL
RESERVES THE RIGHT TO DISALLOW ANY CUSTOMER FROM DELIVERING MCP EXAMS
either based on MCCL findings through secret shopping or from information
provided by ATA that indicates suspicious activity or a breach of exam
security by an ATA testing center. ATA shall ensure that MCCL's such rights
will be stated in the Training Agreement attached to the Master Service
Agreement as Exhibit B of the Master Service Agreement. If MCCL decides to
disallow a Customer for the above reasons, ATA shall take all necessary
actions, without delay, to implement and effectuate such decision of MCCL
10.6 SITE LEVEL STATISTICAL ANALYSIS
ATA will conduct statistical analyses at the site level for MCCL at least
once per fiscal quarter for irregularities such as unusual pass rates or
exam session times, business rule infractions, and other suspicious
patterns in the data. MCCL will make best efforts to provide support for
these analyses.
The purpose of these analyses will be to determine if any testing sites are
passing or failing Candidates at an abnormal rate. These sites are
considered to be "outliers." This analysis will be conducted globally using
statistically sound methodologies (especially using higher volume active
exams). This is done to ensure that the analysis provides enough data to be
statistically sound.
ATA will conduct investigations into these sites and will make
determinations based upon evidence found during investigations, as well as
patterns of abnormal behavior, whether or not action is required. Actions
can range in severity, but could result in permanent closure of the site
(see Section 10.5).
The results of these quarterly investigations will be made available to
MCCL in a report to be included in Appendix E, Report Key.
10.7 INDIVIDUAL CHEATER NOTIFICATION
ATA will work with the Customers and regional channel management groups to
identify all Candidates who are suspected of violating MCP Exam Security
Policy. After notification, ATA shall investigate the situation to
determine if there is enough evidence to substantiate the claim. To the
extent permitted by law, if enough evidence is present, ATA will make best
efforts to prevent the Candidate from taking any future MCP exams. ATA will
then notify MCCL of the infraction, and provide the supporting evidence to
support this decision. ATA will then notify the Candidate via letter of his
or her testing ban, and inform MCCL of the date that letter is sent. Should
Candidates desire to appeal the ban, they will be directed to ATA if the
Candidate feels that the evidence against him/her is neither accurate nor
sufficient, or directed to MCCL if the Candidate admits his/ her guilt and
wants to appeal the severity of the penalty.
ATA will report monthly on all individual cheater activity, with the report
to be included in Appendix E, Report Key.
This individual cheater report shall be shared with all exam delivery
vendors and MCCL regional service centers to ensure that banned Candidates
are kept out of the program (see Section 4.3).
11.0 PROJECT MANAGEMENT
11.1 ATA BRIEFS
MCCL agrees to prepare and use a standard ATA Brief document, which will be
the only method of formal communication with ATA for updates to this SOW,
or for new activities that ATA will accept.
There are two primary types of ATA Briefs, depending on the nature of the
new activity required:
Page 18 of 30
Exhibit A
(1) Program Change Brief --which cover program change requirements, which
incur no additional costs to MCCL, which shall be accepted and
observed by ATA; and
(2) Work-for-Hire Brief - which cover "work-for-hire" activities, for
which additional costs if any, to MCCL must be negotiated and agreed
upon between MCCL and ATA before the start of the activity.
ATA Briefs will be supplied to ATA a minimum of fifteen (15) business
days prior to the start of the activity. ATA will respond to MCCL
within five (5) business days upon receiving ATA Briefs to discuss any
potential issues, including whether a Program Change Brief should
instead be a Work-for-Hire Brief.
In the event that ATA does not respond within five (5) business days,
MCCL and ATA shall assume full compliance with ATA Brief. Above
timelines are subject to change upon the mutual agreement between MCCL
and ATA.
ATA will reject any ATA Brief that is not authorized by signature by
the ATA Account Manager at MCCL or other MCCL contact designated by
same. Any activity implemented without this authorization will be at
ATA's sole financial and resource risk.
11.2 CHANGE MANAGEMENT
MCCL requires that ATA notify MCCL before implementation of a new process
or procedure, or a change to a current process or procedure that affects
MCCL, Microsoft's database or MCP Candidates. MCCL also reserves the right
to approve/disapprove any changes proposed by ATA that affect MCCL or
Candidates; approval shall not be unreasonably withheld.
Upon request, ATA will provide MCCL with copies of their policies or
procedures relevant to the MOE Polytechnic Program as long as they are not
confidential to ATA. Any disclosure of confidential documentation or
information provided to MCCL will be subject to the Non-Disclosure
Agreement then in effect between MCCL and ATA, which is attached hereto as
Appendix J, NDA.
ATA will keep an issue log of all outstanding issues, regarding MOE
Polytechnic Program. The format will be mutually agreed upon by ATA and
MCCL.
12.0 EXAM PUBLISHING, POSTING, AND DISTRIBUTION
12.1 DEFINITIONS
- PUBLISHING (INCLUDES "TECHNICAL DESIGN" AKA "FILE PREP"). The process
of converting raw exam files (e.g., Word, XML, TDL) provided by MCCL
to the XML format needed by the ATA test driver.
- POSTING. The process of placing the XML file within the ATA
registration and delivery systems. This includes setting the
parameters for delivery (e.g., initial registration date, initial
delivery date, required assets).
- DISTRIBUTION. The process by which needed exam files are downloaded to
a Customer in advance of the scheduled test delivery appointment.
12.2 EXAM PUBLISHING, POSTING, AND DISTRIBUTION PROCESS
- ATA shall publish and post exams for MCCL. If MCCL elects to publish
the exams or designate a third-party ("MCCL Exam Designee") to publish
the exams, MCCL shall draft a brief that specifies a mutually agreed
upon publishing process.
- MCCL and ATA will create and agree to a Standard Publishing Schedule,
which will be in the form of a Microsoft Project document (see
Appendix I, Standard Publishing Schedule). Changes to this Standard
Publishing Schedule or deviations from the Standard Publishing
Schedule for a given exam must be agreed to by both parties.
Page 19 of 30
Exhibit A
- MCCL shall make reasonable efforts to limit weekly publishing workflow
to eight (8) exam publications, with no more than three (3) exams on a
given day. With sufficient notice, deviations from this schedule shall
be mutually agreed.
- During the period where ATA is publishing and posting exams for MCCL:
- MCCL will provide ATA with exam publishing and posting schedules
via weekly meetings and by routinely distributing schedule
spreadsheets as needed (minimally, once a month). MCCL shall
provide as much advance notice as possible concerning upcoming
exam publications, which shall typically be at least one month's
advance notice.
- MCCL will forward all necessary files (e.g., exam content using
pre-agreed formats and templates (including Word documents,
graphics, XML,TDL)) and exam specifications (e.g., forms, timing,
scoring, presentation strategy) to ATA for publishing per the
timeframe in the Standard Publishing Schedule.
- ATA will publish the exam and provide a reviewable version to
MCCL for comments and corrections per the Standard Publishing
Schedule (see Appendix I, Standard Publishing Schedule). ATA will
ensure the exam presents and scores properly, but will not be
responsible for content accuracy (unless the content was changed
by reasons on the part of ATA). ATA will not make any content
changes without approval from MCCL.
- MCCL will conduct a quality assurance review of published exams
and return comments to ATA per the Standard Publishing Schedule
(see Appendix J, Standard Publishing Schedule).
- ATA will post final version of exam.
- MCCL shall own all exam source files and shall retain copies of
them. ATA shall also retain copies of all currently live exam
versions. ATA shall retain copies of retired exams, or retired
versions of live exams, for a minimum of two (2) years from the
retirement date, in the published format.
- LOCALIZATION. MCCL requires ATA to support all localized exams
published by MCCL. In addition, MCCL requires that ATA and ATA's
systems support double-byte languages. See also Section 9.3.
12.3 BETA EXAMS
ATA shall deliver MCP Beta exams. These exams use exam series 071 (the
final live MCP exam series is 070). Elements of the Beta process may
include the following:
- MCCL will forward all necessary files to ATA for publishing per the
Standard Publishing Schedule (see Appendix I, Standard Publishing
Schedule), unless shorter timeframes have been mutually agreed.
- MCCL will define beta exam parameters. ATA will open and close beta
exam registration based on these parameters. Parameters may include a
set time frame, a fixed number of registrations, a reduced price or
other promotion, etc.
- ATA will return Beta exam results ("beta rescores") to MCCL as per
normal results reporting processes. MCCL will establish the cut score.
- Beta exams will be re-scored by ATA and results will be sent to
Candidates and MCCL per the Standard Publishing Schedule (see Appendix
I, Standard Publishing Schedule). The objective is to provide final
beta scores to MCCL and Beta exam Candidates prior to Beta exam moving
to live version. Failure to meet the time frame will be cause for ATA
to make reasonable compensation to the affected Candidates, such as
offering a free exam at ATA's expense.
Page 20 of 30
Exhibit A
- Beta exams might include a MCCL-funded promotional incentive in order
to attract Candidates to take the exam during its limited
availability. For example, a promotion might be to issue one or more
exam vouchers for those Candidates who pass the beta exam. Because the
candidates' scores are not determined until several weeks after the
candidate has taken the exam, ATA shall fulfill voucher promotions by
including the voucher(s) with the score report to only those
candidates who qualify for the promotion (e.g., they passed the beta
exam). MCCL shall provide an ATA Brief if ATA is to fulfill
promotional materials other than exam vouchers.
12.4 EXAMS
- ATA shall publish and post exams for the Customers. MCCL reserves the
right to publish the exams or designate a third-party ("MCCL Exam
Designee") to publish the exams.
- During the period where ATA is publishing and posting exams for MCCL:
- MCCL will provide ATA with exam publishing and posting schedules
via regular meetings and by routinely distributing schedule
spreadsheets as needed (minimally, once a month).
- MCCL will forward all necessary files to ATA.
- ATA will post final version of exam
12.5 TECHNICAL DEVELOPMENT
In the case where MCCL or MCCL Exam Designee is developing a new item type
(e.g., simulations, drag-and-drop) or implementing a new test technology
(e.g., computer adaptive testing), upon MCCL's written consent, ATA will
become an active member in the development/psychometric team. MCCL shall
provide a brief and notify ATA a minimum of two (2) weeks prior to the
onset of the project (see Section 11).
The role of ATA member(s) is to ensure that the new item type/test
technology will be fully functional within the ATA test driver. It may be
necessary for members (ATA, MCCL, or MCCL Exam Designee) to travel to
ensure the success of the project. Schedules, travel, and costs shall be
negotiated at the onset of the project.
ATA and MCCL or MCCL Exam Designee shall perform final integration testing
on new item types and new testing technologies within two (2) weeks of the
handoff of final content. Overall project schedules to be mutually agreed
upon prior to implementation of a given project.
13.0 CHANNEL
13.1 CHANNEL AND GEOGRAPHIC COVERAGE
MCCL's goal is to have convenient test delivery and training locations for
Candidates nationwide and establish testing centers in an exam delivery
market segments that do not currently offer MCP exams (e.g., universities).
ATA shall make reasonable efforts to meet or exceed these goals. ATA will
present to MCCL, monthly, ATA's strategy and progress against these goals
and outline all efforts to reach these goals. ATA shall provide MCCL with
copies of executed agreements (Exhibit B of the Master Service Agreement)
and work with MCCL to recruit Customers.
13.2 CHANNEL HARDWARE AND SOFTWARE REQUIREMENT
ATA shall require Customers to conform to at least minimum standards for
hardware and software of its test delivery workstations. These standards
include:
- Display monitors: 17-inch CRT
- Display resolution: 1024x768, 16 bit color
- Workstation: PentiumII(R) CPU, 266MHz or faster
Page 21 of 30
Exhibit A
- Operating system: Microsoft Windows 2000 Professional
- Peripherals: CDROM drive (on at least one workstation per site)
- RAM: 128MB
- Hard drive space: 2GB
- Graphics memory: 8MB
- Virtual memory swap file: 128MB
- Network thru put: 100 MBPS
- Appropriate cameras and other security hardware required to perform
services
ATA shall require training sites to conform to at least minimum standards
for hardware and software for training workstations. These standards are
based on course content as detailed below:
LEVEL 0
- Pentium II 300 MHz
- PCI 2.1 bus
- 256 MB of RAM
- 386 MB of RAM for Virtual PC
- 4-GB hard disk
- 256-KB L2 cache
- DVD player
- Non-ISA network adapter 10/100 Mbps
- 4-MB video adapter
- SVGA monitor (17 inch)
- MS mouse or compatible pointing device
- Sound card with headphones or speaker
LEVEL 1
- Pentium II 400 MHz
- PCI 2.1 bus
- 256 MB of RAM
- 386 MB of RAM for Virtual PC
- 8-GB hard disk
- 256-KB L2 cache
- DVD player
- Non-ISA network adapter 10/100 Mbps
- 4-MB video adapter
- SVGA monitor (17 inch)
- MS mouse or compatible pointing device
- Sound card with headphones or speaker
LEVEL 2
- Pentium III 450 MHz
- PCI 2.1 bus
- 512 MB of RAM
- 640 MB RAM for Virtual PC (recommended)
- 12-GB hard disk
- 512-KB L2 cache
- DVD player
Page 22 of 30
Exhibit A
- Non-ISA network adapter 10/100 Mbps
- 4-MB video adapter
- SVGA monitor (17 inch)
- MS mouse or compatible pointing device
- Sound card with headphones or speaker
LEVEL 3
- Pentium III 700 MHz
- PCI 2.1 bus
- 768 Mb of RAM (same for Virtual PC)
- 16-GB hard disk
- 512-KB L2 cache
- DVD player
- Non-ISA network adapter 10/100 Mbps
- 4-MB video adapter
- SVGA monitor (17 inch)
- MS mouse or compatible pointing device
- Sound card with headphones or speaker
LEVEL 4
- Pentium III 1 GHz
- PCI 2.1 bus
- 1 Gb of RAM (same for Virtual PC)
- 18-GB hard disk
- 256-KB L2 cache
- DVD player
- Non-ISA network adapter 10/100 Mbps
- 4-MB video adapter
- SVGA monitor (17 inch)
- MS mouse or compatible pointing device
- Sound card with headphones or speaker
See Section 13.3 for changes to these minimum requirements.
Page 23 of 30
Exhibit A
13.3 CHANNEL UPGRADES
Changes to Microsoft's exam platforms may require that Customers upgrade
hardware or software used to deliver exams. MCCL and ATA will define a
mutually agreeable timeline as channel upgrades are required. MCCL is under
no financial responsibility for these upgrades. MCCL may consider providing
software at a discounted price to assist in these upgrades. If a site
refuses to comply, MCCL may support ATA's efforts to communicate the
importance of any upgrade to the site. If a site fails to comply with
requests to upgrade, ATA shall prevent that site from delivering the
affected MCP exams, and is permitted to prevent that site from delivering
all MCP exams, until the appropriate upgrade is completed.
13.4 CHANNEL REPORTING
ATA shall produce the following channel reports per Appendix E, Report Key:
- On-line, real-time report of all active Customers
- On a semiannual basis, or other mutually agreeable timeframe, ATA will
produce a geographic map displaying where Customers are located using
Microsoft MapPoint or other agreed-upon format.
- A monthly summary of all test centers by region.
14.0 ATA MARKETING ACTIVITIES
ATA is encouraged to perform marketing activities to not only increase the
market effect of its delivery of MCP exams, but to grow the overall MCP
certification business. Marketing activities may include, but are not
limited to: print and Web site advertising; trade-show exhibitions,
sponsorships, and on-site testing; and direct mail (e-mail and postal
mail). However, during its marketing activities ATA must follow Microsoft's
trademark use rules, which are provided in the Master Service Agreement and
this SOW.
ATA shall provide a Marketing Plan for approval by MCCL by June 30, 2003,
and thereafter, every six (6) months. This Plan shall include budgets and
activities ATA plans to perform to grow MCP business. MCCL shall provide
timely feedback on the Marketing Plan and shall not unreasonably withhold
approval.
MCCL reserves the right to prohibit ATA from direct-marketing to customers
whose names are obtained from MS-Cert and MCP candidate databases. This
consent to direct market will not be unreasonably withheld.
All direct marketing must consider candidate privacy issues, and must
therefore exclude any candidates who have indicated they do not want
marketing correspondence. (See Section 7.5, Data Confidentiality and
Privacy Guidelines).
14.1 MCCL SPECIAL PRICING
From time to time MCCL reserves the right to provide special pricing for
Courseware Materials and test bundles or other co-marketing programs
mutually agreed upon. These, if any, will be defined in separate ATA
Briefs.
14.2 VOUCHER SYSTEMS
- ATA VOUCHER SYSTEM. ATA may choose to create and support a paper or
electronic voucher system of ATA's design in order to facilitate exam
promotions and/or discounts mutually agreed upon by MCCL and ATA.
ATA's system shall track voucher usage in order to ensure that a
single voucher is not used more than once for MCP exams. MCCL shall
determine expiration length of the vouchers and ATA shall print the
appropriate expiration date on each voucher. Exams must be taken by
the expiration date on the voucher. There is no requirement at this
time that ATA should honor exam vouchers from another MCP exam vendor,
however, MCCL has the option to change this requirement at a future
date. ATA confirms that such voucher shall not affect ATA's obligation
to remit Licensing Fees to MCCL.
14.3 DISCOUNT REPORTS
Page 24 of 30
Exhibit A
As appropriate, ATA will create and submit any and all Discount reports
listed in Appendix E, Report Key.
14.4 VOUCHER EXPIRATIONS
Pre-paid exam vouchers shall be valid for no more than six (6) months
unless otherwise authorized by MCCL. ATA shall have the ability to extend
the expiration date of outstanding (unexpired) vouchers, but will not do so
without express authorization from MCCL.
14.5 GENERAL VOUCHER RULES
- A Candidate may redeem no more than one (1) voucher per exam. A single
voucher may not be redeemed on more than one exam.
- If a voucher is redeemed for an exam whose price is less than the cost
of the voucher ("underutilized vouchers"), then the Candidate forfeits
any remaining balance.
- A promotional discount voucher (e.g., 25% discount) may not be
combined with other promotional discounts.
- A voucher's expiration date may not be effectively-extended by
rescheduling the exam delivery date past the voucher's expiration
date.
- Vouchers must include Candidate's name and are non-transferable
15.0 BUSINESS REVIEW MEETINGS
ATA agrees to attend regular Business Review meetings and present according
to an agenda as mutually agreed upon with MCCL in advance. These Business
Review meetings should take place monthly, or a mutually agreed timeframe.
At MCCL's discretion, ATA will also need to present on a Regional basis at
specified MCCL Regional locations. ATA will prepare presentations using
Microsoft PowerPoint software to enable MCCL to read the presentation in
soft copy as well as hard copy. These Business Review meetings will
typically take place at the office of MCCL, although Business Review
meetings may take place at the office of ATA or via teleconference at
MCCL's discretion.
16.0 ACCOUNT MANAGEMENT AND CLIENT SERVICES TEAM
ATA will supply the following points of contact for MCCL. Most of these
contacts do not have to be dedicated to MCCL, but they must be known to
MCCL and responsive to MCCL, and if the people in these roles change, MCCL
must be notified of the change.
- ACCOUNT EXECUTIVE. This person should be responsible, at a high level,
for all aspects of the relationship between ATA and MCCL.
Responsibilities shall include, but are not limited to: contract
issues, relationship issues, overall service quality, and acting as a
point of escalation for the entire MCP exam service team.
- PROGRAM MANAGER. This person should be dedicated to the MCCL account
and should have a more intimate, working knowledge of the MCCL-ATA
relationship. Responsibilities shall include, but are not limited to:
exam development and publishing, call center, finance, data feeds and
systems, and resolving open issues with the day-to-day management of
the MOE Polytechnic Program as it is delivered by ATA. The Program
Manager should have sufficient level within ATA to make decisions and
direct the appropriate support people. This point of contact shall be
expected to meet (via conference call) with the ATA Account Manager at
MCCL and the MCP Exam Development Team or other MCCL contacts at least
once a week.
- TECHNICAL CONTACT(S). This person(s) should have intimate knowledge of
ATA's data systems and exam development and publishing. This person
should be able to work effectively with MCCL on new technologies in
exam development and the data systems that support those exams.
- EXAM PUBLISHING CONTACT(S). This person(s) should be responsible for
all aspects of exam publishing.
Page 25 of 30
Exhibit A
17.0 REPORTING
ATA will produce all reports requested in this SOW and listed in Appendix
E, Report Key and Appendix L, ATA Program Reconciliation Reports. Samples
of all reports are in Appendix E, Report Key and Appendix L, ATA Program
Reconciliation Reports. All reports must be sent to MCCL electronically and
produced in a Microsoft product such as Excel, Word, or PowerPoint. MCCL
reserves the right to request any additional reporting necessary as either
regular, standard reports or on an ad-hoc basis, or may request changes to
the reports listed in this section. Reasonable requests for new reports or
changes to existing will be defined in an ATA Brief as necessary.
ATA is not responsible for meeting all KPIs listed in Appendix A, KPI
Reporting Requirements, but rather those listed here:
- Phone service level (by service center) (See Section 3.2)
- Percentage of Calls Handled (by service center) (See Section 3.2)
- Exam service level (See Section 9.1)
- Exam data sent (See Section 8.6)
- Number of calls answered (by service center) (See Section 3.2
- Number of registrations taken at call centers (See Section Exams
Registered by Phone in Appendix A attached hereto)
- Number of registrations taken via the Internet (See Section Exams
Registered by Internet in Appendix A attached hereto)
- Number of registrations taken at testing centers (See Section Exams
Registered On-Site in Appendix A attached hereto)
- Number of exams delivered. (See Section Exams Delivered in Appendix A
attached hereto)
18.0 FINANCIAL OVERVIEW
ATA will provide financial reporting on a monthly basis as part of the
obligations by ATA towards MCCL. ATA is fully responsible for all ATA tax
liabilities (to include collection and remittance to the appropriate tax
authorities), bad debt risk, and all risks associated with repatriation of
money from affiliated locations. Failure to follow the requirements listed
in this section, including, but not limited to, any failure to deliver
compliant, accurate and timely reporting may be deemed breach of contract
and MCCL and ATA may seek any or all remedies under the Master Service
Agreement.
18.1 ATA LICENSING FEES:
ATA shall pay to MCCL Licensing Fees for MCCL's license to ATA, as
specified in Section 1.0 hereinabove. The amount of such licensing fees
shall be due for each Candidate as described in ATA Price List (see
Appendix F, ATA Price List) (the "Licensing Fees).
ATA shall remit to MCCL the full amount of the Licensing Fees, without any
types of deduction or offset (including deduction for tax purposes).
If ATA chooses to offer a discount promotion for a MCP exam, ATA shall do
so at ATA's expense and without affect to MCCL's remittance.
The payment of the Licensing Fees is provided for in Section 20.1
hereinbelow.
18.2 OTHER BENEFITS
MCCL will fund the following benefits to ATA through the earlier of June
30, 2003 or until the Master Service Agreement is terminated:
- Co-marketing funds as described in Section 14.1.
Page 26 of 30
Exhibit A
18.3 FORFEITURE REPORTS
ATA will produce a Forfeiture Report that supports the amount of forfeited
exam funds remitted to MCCL. This report will detail those exams that are
prepaid and were neither taken nor refunded. This report will also detail
each Candidate who has paid and registered for an exam but did not take the
exam and did not cancel the exam as described in Sections 4.6. This report
should list each unredeemed exam registration number for "No Shows" and by
PO number or voucher series (as appropriate) number for prepaid exams.
Purchaser name and contact information shall also be included. See Appendix
E, Report Key, for a sample of this report.
18.4 EXPIRED EXAMS
If ATA implements a business rule that limits the maximum period between
initial exam registration and exam administration, and this rule results in
a Candidate forfeiting fees, those forfeited funds will be treated as
expired vouchers and the funds will be remitted to MCCL. ATA shall report
this activity via the Forfeiture Report, per Appendix E, Report Key.
18.5 PRICING SCHEDULE MAINTENANCE
MCCL has created a price schedule, referred to as ATA Price List (Appendix
F), which lists all royalties for products offered through the MOE
Polytechnic Program. MCCL reserves the right to make any change(s) to
Appendix F, ATA Price List at MCCL's own discretion. MCCL will communicate
all changes to ATA thirty (30) calendar days prior to the date that the
change(s) are to be implemented by ATA.
19.0 COLLECTIONS BY ATA
19.1 ACCEPTABLE PAYMENT METHODS
ATA will determine acceptable payment methods from each and all of the
Customer (such as transactions with credit cards, bank orders and checks,
purchase orders, etc.). MCCL accepts no liability for unpaid debts if ATA
chooses to extend terms.
19.2 REMITTANCE TO MCCL
Regardless of ATA's ability to collect fees from each and all of the
Customers, MCCL shall receive full remittance of the Licensing Fees as set
forth in this SOW.
20.0 FINANCIAL REMITTANCES, REPORTS, AND INVOICES
AT THE END OF EACH FINANCIAL MONTH ATA WILL PROVIDE MCCL WITH FULL
REPORTING OF ALL ACTIVITIES AS LISTED IN APPENDIX L, ATA REVENUE
RECONCILIATION REPORTS. ALL SUCH REPORTING WILL BE PROVIDED IN THE FORM OF
AND EXCEL WORKBOOK. ALL REPORTS SHALL BE TIMELY, COMPLETE AND ACCURATE.
20.1 INVOICING
ON RECEIPT OF THIS INFORMATION MCCL WILL GENERATE AN INVOICE FOR THE TOTAL
LICENSING FEES DUE BASED ON THE NUMBER OF UNITS OF PRINTED COURSEWARE
MATERIALS SHIPPED BY ATA DURING THE MONTH AND MCCL SHALL ISSUE THIS INVOICE
TO ATA. ATA WILL BE FULLY RESPONSIBLE FOR AND LIABLE TO MCCL FOR THE
PAYMENT IN FULL OF THE DUE AMOUNT. MCCL WILL NOT BE RESPONSIBLE FOR ANY
INVOICES INCURRED BY ATA AND ATA'S PARTNERS.
20.2 WELCOME KITS
At the end of each financial month ATA will provide details to MCCL of the
quantity of 'Welcome Kits' shipped, as listed in Appendix L, ATA Revenue
Reconciliation Reports.
21.0 CERTIFICATES
Page 27 of 30
Exhibit A
ATA shall perform at its own cost the following services in compliance with
specifications mutually agreed upon by ATA and MCCL. Changes to
requirements and specifications will be made through ATA Briefs as referred
to in Section 11.
21.1 EXAM CERTIFICATES
For each course offered in the program with a MCP exam, ATA will provide
Candidates who have successfully completed the course and exam an exam
certificate, the form of which shall be determined by MCCL and may be
updated from time to time at MCCL's sole discretion.
The exam certificates will include at a minimum, the course name, student
score report, security features and other features as mutually agreed upon
by MCCL and ATA. Candidates will receive exam certificates for each course
and exam completed.
ATA must provide sample exam certificates to MCCL for approval prior to
delivery to Candidates.
21.2 COURSE COMPLETION CERTIFICATES
For MCP courses offered under the MOE Polytechnic Program without a MCP
exam, ATA will provide Candidates who have successfully completed the
course with a course completion certification, the form of which shall be
determined by MCCL and may be updated from time to time at MCCL's sole
discretion.
The course completion certificate will include at a minimum the course
name, student name and other features as mutually agreed upon by MCCL and
ATA.
ATA must provide sample course completion certificates to MCCL for approval
prior to delivery to Candidates.
21.3 CERTIFICATION WELCOME KITS
For Candidates who have successfully completed their first MCP exam or
reach an advanced certification such as MCSE, MCSA, MCDBA, MCAD, MCSD or
other certifications specified by MCCL, ATA shall provide each of such
Candidates a welcome kit.
The contents of the welcome kit will include at a minimum, a certificate,
MCP program guide, welcome letter and other features as mutually agreed
upon by MCCL and ATA.
ATA must provide sample welcome kits to MCCL for approval prior to
shipments to Candidates.
22.0 LIST OF APPENDIXES
Appendix A. KPI Reporting Requirements
Appendix B. MCP Customer Service Calls
Appendix C. Missing Exam Information and Data Feed Process
Appendix D. ATA Exam Results Import Formats
Appendix E. Report Key
Appendix F. ATA Price List
Appendix G. Independent Contractor Agreement
Page 28 of 30
Exhibit A
Appendix H. MCCL Fiscal Calendar, FY/2003-4
Appendix I. Standard Publishing Schedule
Appendix J. NDA
Appendix K. MCP Exam Provider Logo Usage Guidelines
Appendix L. ATA Revenue Reconciliation Reports
23.0 INCORPORATION
This SOW and its Appendices attached hereto are an indispensable part of
the Master Service Agreement and further provide the rights and obligations
of MCCL and ATA. All the terms and conditions of the Master Service
Agreement, unless further specified herein, are therefore deemed as having
been incorporated into this SOW and its Appendices and shall apply to this
SOW and its Appendices.
Page 29 of 30
Exhibit A
IN WITNESS WHEREOF, MCCL and ATA have entered into this SOW on the date
first set forth above.
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD.
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
Page 30 of 30
APPENDIX A TO EXHIBIT A
This Key Performance Indicators ("KPIs") is attached hereto as Appendix A to the
Exhibit A Statement of Work ("SOW") of the Master Services Agreement entered
into by and between (CHINESE CHARACTERS) Microsoft (China) Co., Ltd. and
(CHINESE CHARACTERS) ATA TESTING AUTHORITY, BEIJING CO., LTD. ("ATA") on May 16,
2003. This KPIs may be updated by Microsoft from time to time and ATA shall obey
the KPIs in accordance with the Master Services Agreement and SOW.
The term "customer" and "candidate" provided in this KPIs shall have the same
meaning as the "Candidate" as defined in the SOW.
APPENDIX A TO EXHIBIT A
(WORLDWIDE OPERATIONS LOGO)
KPI REPORTING REQUIREMENTS
WORLDWIDE PROGRAMS
KEY PERFORMANCE INDICATORS - KPIS
Glossary
1
Worldwide Programs - KPIs GLOSSARY
REVISION 1 - SUMMARY
INTRODUCTION
Below is a summary of the changes made to the KPI Glossary. It is strongly
recommended that vendors review the sections and specific metrics that are
specific to their businesses to ensure that they understand the definitions and
any changes that are not described in detail below. This is also true for the
KPI Reporting Requirements document as there are changes to the region hierarchy
and Measurement Codes.
EXTERNAL CUSTOMER SATISFACTION (not applicable to vendors)
The KPI for this measurement is no longer based on a single question about the
customer's satisfaction with the overall service received from Microsoft. WPO's
external customer satisfaction measurement is now based on an average of 4 "rep
attributes".
ORDER FILL RATE
This metric only measures "compliant" order lines, i.e. those order lines that
include all of the necessary information for the product to be fulfilled. In the
new definition, orders for product that have not been released to manufacturing
are not compliant. Therefore the measurement does not reflect RTM issues,
however Order Fill Rate does continue to be impacted by backorders resulting
from poor forecasting by Microsoft or the vendor, supplier problems, or other
issues within WPO's control.
PERFORMANCE CONSOLIDATION
Microsoft consolidates performance to the region and often the WPO level
(Example: total Order Fill Rate for North America across all programs). To
ensure that this consolidation is calculated correctly, Microsoft needs the
common denominator used to calculate the metric in addition to the metric
itself. For example, Phone Service Level is the number of calls answered within
a specified time period in relation to the total number of calls offered.
Microsoft needs the total number of calls offered to consolidate one program's
performance with that of other programs. To accomplish this Calls Offered and
Compliant Order Lines Received have been added as new metrics. These are
mandatory for vendors reporting Phone Service Level and Order Fill Rate
respectively.
DEVELOPER METRICS
This group of measurements that are specific to the MSDN and TechNet programs
have been added to the KPI Glossary, although many vendors already report these
today. Some Developer Metrics have been added and others discontinued. Please
recognize that an important change has been made as these metrics measure the
number of subscriptions and not the number of subscribers as was the case with
some of these metrics in the past.
OTHER NEW METRICS
The following metrics have been added to the glossary: Average Hold Time,
Compliant Order Percentage, Calls Answered - IVR, Outbound Calls, Average Calls
Per Day, Order Fill Rate - MS Dictated Hold, Longest Time to Order Fill,
Shipping Issues as % of Carrier Shipments - Due, Beta CD Volumes, MSN Volumes
and some Exam Metrics and Volumes. Please reference the Required Metrics Matrix
to find out which of these apply to each vendor.
2
DELETED METRICS
The following metrics were deleted from the glossary or the original Developer
Metrics definitions: Obsolescence Units, Events Metrics, End Item SKUs Handed
Off, BOMs Frozen X Days Late, RTM Complete X Days Late, Forecast Release X Days
Late, Forecasted Quantities by End Item SKU, Manufactured Quantities by End Item
SKU, Ordered Quantities by End Item SKU, Disks Replicated, Paid Subscriptions,
and Subscriber Retention.
LEGEND
On page 8 is a legend to better explain the glossary's format and language.
3
Table of Contents
4
5
6
LEGEND
PURPOSE--The "Purpose" describes what activity or process Microsoft is measuring
with the respective metric.
DEFINITION
- The "Definition" is a short explanation of the metric and how it
should be measured.
CALCULATION
The "Calculation" is a mathematical representation of the metric and
exactly how it should be calculated.
- Additional information about the calculation and definitions for terms
used in the calculation are outlined in this section.
FREQUENCY--The "Frequency" describes how often this metric should be reported to
Microsoft. Usually, this distinguishes whether the metric should be reported
both weekly and monthly or just monthly. Activity should always be reported for
Microsoft's fiscal periods.
BENCHMARK - The "Benchmark" represents Microsoft's desired service level. For
metrics that are reported as a percentage, the Benchmark will often dictate the
denominator in the calculation. For Example: Phone Service Level measures the
number of calls answered within a specified time period. This specified time
period is described in the Benchmark. Many metrics' Benchmarks vary by region
and program. In these cases the Benchmark should be communicated by the
Microsoft Vendor Account Manager. Metrics that measure volumes and not
performance do not have Benchmarks.
MEASUREMENT TRAPS
- "Measurement Traps" describe incorrect ways to measure the respective
metric. When calculating the metric the vendor should ensure that they
are NOT following any of these practices.
OTHER NOTES (not part of the glossary format)
- WPO = Worldwide Programs Operations
- MS = Microsoft
7
TOP KPIS - PERFORMANCE
8
EXTERNAL CUSTOMER SATISFACTION WITH REP
PURPOSE - Measure external customers' satisfaction with the quality of service
received during a call to Microsoft
DEFINITION
- SATISFIED: The percentage of respondents whose average score indicates
satisfaction.
- DISSATISFIED: The percentage of respondents whose average score
indicates dissatisfaction.
CALCULATION
- SATISFIED:
Number of respondents whose average score from the rep attribute
questions is 7 or greater
Number of respondents to complete a survey
- DISSATISFIED:
Number of respondents whose average score from the rep attribute questions
is 3 or lower
Number of respondents to complete a survey
- This metric measures customers' responses to the following rep
attributes:
- Call Resolution
- Professionalism/Communication Skills
- Product/Program Knowledge
- Speed of Service (Hold Time/Rep Efficiency)
These rep attribute questions ask specifically about processes that
are managed by WPO and its vendors (rep performance, etc.). This
metric should not measure responses about program content, business
rules established by the program owner, or other elements outside of
WPO's control.
- Customers are asked to respond to the rep attribute questions on a 9
point scale. "9" indicates that they completely agree that the
statement describes their experience and "1" indicates that they do
not agree at all.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY-- MS Fiscal Month
BENCHMARK - Satisfied: 85%, Dissatisfied 5%
MEASUREMENT TRAPS
- Scoring on questions other than the rep attributes selected to measure
WPO's performance.
9
- Not using a 9 point scale.
- Allowing vendors to pre-select or filter the list of customers before
they are given to the survey company.
10
PHONE SERVICE LEVEL
PURPOSE--Measure phone response time related to live agents
DEFINITION
- The percentage of inbound calls answered by a live rep within "y"
seconds of the time the call enters the live rep queue in relation to
the number of total calls offered to the live rep queue.
CALCULATION
Number of calls answered by a live rep within "y" seconds of entering
the live rep queue
Total number of calls offered to the live rep queue
- Live rep queue = post-messaging and post-IVR
- Total number of calls offered includes all abandons within the live
rep queue. See the Calls Offered metric definition.
- Should be system generated via ACD software or a consistent process
that is auditable
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x% in y seconds (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Dividing by total calls answered, rather than total calls offered.
- Allowance for abandons in first "y" seconds within the live rep queue,
rather than including all abandons within the live rep queue.
- Adding estimated time for messaging to contractual target, rather than
starting clock at live rep queue and measuring pure response time.
- Including IVR response time in numerator, rather than live rep queue
only.
- Including IVR total calls offered in denominator, rather than live rep
queue only.
- Including voicemail response in numerator
11
ORDER FILL RATE
PURPOSE-- Measure overall WPO fulfillment performance. This represents WPO's
ability to accept orders and fulfill product in a timely manner per specific
business requirements.
DEFINITION
- The percentage of total compliant order lines fulfilled within y days
- This includes orders that were delayed or backordered as a result of
WPO or its vendors.
CALCULATION
Number of compliant order lines received workday 0 and
shipped within "y "days
Number of compliant order lines received workday 0
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- For those vendors that report more than one of the following Order
Fill Rate metrics, this metric should represent a weighed average of
those metrics:
- Order Fill Rate - One-Off or Welcome-Kit
- Order Fill Rate - Subscription/Monthly Mailer
- Order Fill Rate - Expedite/Rush Orders
- Order Fill Rate - Replacement Parts/Reships
- Order Fill Rate - Microsoft Dictated Hold
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK -x % in y days. (see statement of work or Microsoft Vendor Account
Manager)
- Order lines received workday 0 and shipped workday 0 = same day.
- Order lines received workday 0 and shipped workday 1 = 1 day
MEASUREMENT TRAPS
- Excluding order lines that were not shipped because the product was
backordered, i.e. poor forecasting or problems with a supplier
- Measuring workday 0 from the packslip print instead of actual order
entry
12
- Measuring workday 0 from the third-party feed to fulfillment center,
rather than capturing actual front-end order date
13
DISTRIBUTION LEAD TIME PERFORMANCE
PURPOSE-- Measure timeliness of delivery from shipment pick-up to consignee
DEFINITION
- Percentage of shipments delivered to consignee within the quoted
distribution time, including ACTUAL domestic transit, international
transit, carrier performance, and customs factors.
- Consignee = "ship to" party as reflected on the airway bill
CALCULATION
Number of carrier shipments that are due within the reporting period
and WERE DELIVERED TO THE CONSIGNEE WITHIN THE QUOTED DISTRIBUTION TIME
Total number of carrier shipments due to the consignee during the reporting
period
- See Carrier Shipments - Due definition page.
- Distribution Lead Time Performance should be reported based on due
date. Although carrier shipments may have been delivered prior to the
reporting period in which they were due, they should be measured and
reported based on due date.
- This should be calculated at the lowest level of available detail in
regards to program and region.
- The Quoted Distribution Time is established by Microsoft.
DISTRIBUTION FLOW
Dist Whse > Dom Trans > Intl Gateway > Intl Trans > Intl Customs > Dom Trans >
Consignee
- - Distribution Warehouse = point at which shipment originates
- - Domestic Transportation = transit from fulfillment warehouse to
international port of export/gateway
- - International Gateway = point where shipment/product is processed/cleared
through local customs
- - International Transportation = transit from international port of
export/gateway to international port of entry
- - International Customs = transition point at port of entry where title and
liability of shipment transfers from the shipper to the consignee
- - Domestic Transportation in-country = delivery to end user/consignee
- - Consignee = "ship to" party as reflected on the airway bill
FREQUENCY--MS Fiscal Month
BENCHMARK - 95% of shipments delivered to the consignee within the quoted
distribution time
MEASUREMENT TRAPS
- - Measuring transit time only, rather than full distribution process.
14
- - Measuring delivery to customs or an alternate consignee rather than
delivery to the consignee as reflected on the airway bill.
- - Measuring shipments picked-up or delivered during a reporting period rather
than those due during the reporting period.
15
OBSOLESCENCE
PURPOSE-- Measure forecasting accuracy through Microsoft authorized scrap
DEFINITION
- Total dollars related to measurable units scrapped and obsolete
inventory in relation to total standard product costs associated with
all products produced.
CALCULATION
Obsolete and Scrapped Inventory Expense
Total Product Costs
- This measurement is calculated using information provided from the WPO
Business Stream, by stream and region.
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Quarter - Year to Date
BENCHMARK - 4%
MEASUREMENT TRAPS
- Excluding components
- Including non-authorized units
- Reporting in local currency
- Excluding disposal cost, transportation, etc.
16
TOP KPIS - VOLUMES
17
CALLS ANSWERED
PURPOSE-- Measure volume of inbound calls.
DEFINITION
- Inbound calls answered by a live rep and inbound calls completed in an
IVR
CALCULATION
Count of inbound calls answered by a live rep + inbound calls
completed in an IVR
- If an IVR is not used, then this metric becomes purely a count of
inbound calls answered by a live rep.
- Calls completed in an IVR represent calls where the customer chose at
least one option in the IVR routing system. This does not include
calls that were abandoned during the first routing message. For voice
mail systems that do not support multiple routing options a call that
generates a message is considered "answered"; a call that does not is
abandoned and not counted in this metric.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting calls offered rather than calls answered
- Including outbound calls, rather than inbound only
- Double counting internal transfers/escalations
- Excluding IVR calls completed
- Counting calls that were abandoned during or before the first routing
message in an IVR
18
CALLS OFFERED
PURPOSE-- Measure volume of inbound calls offered to the live rep queue
DEFINITION
- Inbound calls offered to the live rep queue
CALCULATION
Count of inbound calls offered to a live rep queue
- Calls offered represents all calls, answered or not. This should be
measured from the earliest indication that a call has reached the live
rep queue.
- This should be the same measurement that is used to calculate Phone
Service Level and other phone metrics that use "calls offered".
Microsoft will use this information to aggregate phone service level
performance.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting calls answered rather than calls offered
- Including outbound calls, rather than inbound only
- Double counting internal transfers/escalations
- Including IVR calls offered
- Excluding calls that were abandoned during or before the live rep
queue
19
PHONE ORDERS ENTERED
PURPOSE-- Measure volume of orders received via phone or IVR.
DEFINITION
- Orders received via phone or IVR and entered into an order entry
system.
CALCULATION
Count of compliant orders received via phone and IVR and entered into
an order entry system.
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Excludes cycle pulls or downloads for subscription welcome kits,
monthly mailers, or pid activations
- Includes supplemental and replacement parts for subscription programs
- This should be calculated at the lowest level of available detail in
regards to program and region.
- For Microsoft developer programs, MSDN and TechNet, this includes any
transactions that trigger payment or subscriber information
processing. Examples: Renewals, Upgrades, Cancellations, Activations,
Pack-Ship Changes, Re-Ships, and Returns.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting orders shipped, rather than orders entered
- Excluding IVR orders entered
- Counting non-compliant orders
- Counting cycle pulls or downloads for subscription programs
MAIL/FAX ORDERS ENTERED
20
PURPOSE-- Measure volume of orders received via mail or fax.
DEFINITION
- Orders received via mail or fax and keyed into order entry system
CALCULATION
Count of compliant orders received via mail or fax and entered
into order entry system
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Excludes cycle pulls or downloads for subscription welcome kits,
monthly mailers, or pid activations
- Includes supplemental and replacement parts for subscription programs
- This should be calculated at the lowest level of available detail in
regards to program and region.
- For Microsoft developer programs, MSDN and TechNet, this includes any
transactions that trigger payment or subscriber information
processing. Examples: Renewals, Upgrades, Cancellations, Activations,
Pack-Ship Changes, Re-Ships, and Returns.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting orders shipped, rather than orders entered
- Counting mail/fax orders received, rather than mail/fax orders entered
- Counting non-compliant orders
- Counting cycle pulls or downloads for subscription programs
21
INTERNET ORDERS ENTERED
PURPOSE-- Measure volume of orders submitted via the internet/web.
DEFINITION
- Orders submitted via internet/web and downloaded/keyed into order
entry system
CALCULATION
Count of compliant orders received via internet/web and
downloaded/keyed into order entry system
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Excludes cycle pulls or downloads for subscription welcome kits,
monthly mailers, or pid activations
- Includes supplemental and replacement parts for subscription programs
- This should be calculated at the lowest level of available detail in
regards to web site, program and region.
- For Microsoft developer programs, MSDN and TechNet, this includes any
transactions that trigger payment or subscriber information
processing. Examples: Renewals, Upgrades, Cancellations, Activations,
Pack-Ship Changes, Re-Ships, and Returns.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting orders shipped, rather than orders entered
- Counting web/internet orders as download, rather than as internet
orders
- Counting email orders as internet orders, rather than as email orders
- Counting non-compliant orders
- Counting cycle pulls or downloads for subscription programs
EMAIL ORDERS ENTERED
22
PURPOSE-- Measure volume of orders received via email.
DEFINITION
- Orders received via email and downloaded/keyed into order entry system
CALCULATION
Count of orders received via email and downloaded/keyed into
order entry system
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Excludes cycle pulls or downloads for subscription welcome kits,
monthly mailers, or pid activations
- Includes supplemental and replacement parts for subscription programs
- This should be calculated at the lowest level of available detail in
regards to web site, program and region.
- For Microsoft developer programs, MSDN and TechNet, this includes any
transactions that trigger payment or subscriber information
processing. Examples: Renewals, Upgrades, Cancellations, Activations,
Pack-Ship Changes, Re-Ships, and Returns.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting orders shipped, rather than orders entered
- Counting email orders as internet orders, rather than as email orders
- Counting non-compliant orders
- Counting cycle pulls or downloads for subscription programs
DOWNLOAD ORDERS ENTERED
23
PURPOSE-- Measure orders entered received via tape/feed from non-internet.
DEFINITION
- Orders received via tape/feed from non-internet source and
downloaded/entered into order entry system
CALCULATION
Count of orders received via download from non-internet source and
downloaded/entered into order entry system
- Excludes cycle pulls or downloads for subscription welcome kits,
monthly mailers, or pid activations
- Includes supplemental and replacement parts for subscription programs
- This should be calculated at the lowest level of available detail in
regards to program and region.
- For Microsoft developer programs, MSDN and TechNet, this includes any
transactions that trigger payment or subscriber information
processing. Examples: Renewals, Upgrades, Cancellations, Activations,
Pack-Ship Changes, Re-Ships, and Returns.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting orders shipped, rather than orders entered
- Counting web/internet orders as download, rather than as internet
orders
- Counting cycle pulls or downloads for subscription programs
24
TOTAL ORDERS ENTERED
PURPOSE-- Measure volume of total orders entered
DEFINITION
- Sum of all order types. Phone + Mail/Fax + Internet + Email +
Download.
CALCULATION
Sum of all compliant orders entered
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Excludes cycle pulls or downloads for subscription welcome kits,
monthly mailers, or pid activations
- Includes supplemental and replacement parts for subscription programs
- This should be calculated at the lowest level of available detail in
regards to web site, program and region.
- For Microsoft developer programs, MSDN and TechNet, this includes any
transactions that trigger payment or subscriber information
processing. Examples: Renewals, Upgrades, Cancellations, Activations,
Pack-Ship Changes, Re-Ships, and Returns.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting orders shipped, rather than orders entered
- Excluding an order type
- Counting non-compliant orders
25
UNITS SHIPPED TO END-CUSTOMER
PURPOSE-- Measure the quantity of units shipped to an End-Customer.
DEFINITION
- Total quantity of confirmed End Items shipped to End-Customers
CALCULATION
Sum of shipped quantities by end item SKU that are shipped directly to an
End-Customer
- End-Customer represents an enduser or an MS subsidiary at which WW
Programs has fulfilled its obligation to fulfill the product.
End-Customer does not include a vendor's regional distribution
facility from which the product is shipped to endusers.
- End Items are measured by Financial End Item SKU.
- This measurement represents the transaction where revenue is
recognized.
- A reship counts as 1 end item SKU shipped
- This should be calculated at the lowest level of available detail in
regards to campaign, country and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK -n/a
MEASUREMENT TRAPS
- Reporting by SKU, rather than summary of all SKUs
- Summing at a level different than end item SKU
- Excluding reships, comped shipments, replacement parts, supplemental
parts, marketing parts from count
- Reporting units at a program level and not broken out by campaign
- Including shipments to regional vendor distribution facilities
COMPLIANT ORDER LINES RECEIVED
26
PURPOSE-- Measure the quantity of compliant order lines received.
DEFINITION
- Total quantity of compliant order lines received.
CALCULATION
Sum of compliant order lines received that generate a fulfillment request
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- This should be the same measurement that is used to calculate Order
Fill Rate and other fulfillment or front-end metrics that use
"compliant order lines". Microsoft will use this information to
aggregate fulfillment performance.
- This should be calculated at the lowest level of available detail in
regards to campaign, country and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK -n/a
MEASUREMENT TRAPS
- Reporting the number of orders and not order lines
- Reporting non-compliant order lines
27
CARRIER SHIPMENTS - DUE
PURPOSE-- Measure the volume of shipments due to the consignee.
DEFINITION
- Count of shipments due to the consignee during the reporting period.
CALCULATION
Count of shipments due to the consignee during the reporting period
- Shipment = Single airway bill
- Due date is dictated by the quoted distribution time for the
individual shipment
- Includes shipments to a single consignee and bulk shipments to
Microsoft subsidiaries, distis, etc.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting end item SKUs, rather than shipments as measured by airway
bills.
- Counting shipments picked-up, rather than shipments due.
- Counting shipments delivered, rather than shipments due.
28
ACTIVE SUBSCRIPTIONS
PURPOSE-- Measure volume of active subscriptions in a program.
DEFINITION
- The number of active subscriptions in the program at a point in time
CALCULATION
Count of active subscriptions as of Friday of each fiscal week or month
- This should only be calculated for programs that have a subscriber or
member base. It should not be calculated for pick/pack/ship programs
on endusers serviced.
- This should be calculated at the lowest level of available detail in
regards to program and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Applying this metric to pick/pack/ship programs
- Counting new subscriptions during the month, rather than total active
subscriptions at a point in time
- Not accounting for cancelled or expired subscriptions
- Not giving country visibility
29
SUPPORTING METRICS - FRONT END PERFORMANCE
30
ABANDON RATE - ABSOLUTE
PURPOSE-- Measure calls lost or dropped during any part of the call process
DEFINITION
- The percentage of inbound calls that are lost or dropped in the
messaging system, IVR, or live rep queue before the caller receives a
live response or has been serviced by an IVR option in relation to the
number of total calls.
CALCULATION
Number of calls abandoned from live rep queue + messaging + IVR
Total number of calls offered
- Should be system generated via ACD software or a consistent process
that is auditable
- This should be calculated at the lowest level of available detail in
regards to program, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Dividing by total calls answered, rather than total calls offered.
- Excluding hangups in first x seconds.
- Counting calls abandoned in live rep queue only.
- Counting only total calls offered to the live rep queue in
denominator, rather than all calls offered.
31
ABANDON RATE - LIVE REP QUEUE
PURPOSE-- Measure calls lost or dropped in live rep queue prior to pickup by a
live agent.
DEFINITION
- The percentage of inbound calls that are lost or dropped in the live
rep queue before the caller receives a live response in relation to
the number of total calls offered to the live rep queue.
CALCULATION
Number of calls abandoned in the live rep queue
Total number of calls offered to the live rep queue
- Live rep queue = post-messaging and post-IVR
- Should be system generated via ACD software or a consistent process
that is auditable
- This should be calculated at the lowest level of available detail in
regards to program, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - 4%
MEASUREMENT TRAPS
- Dividing by total calls answered, rather than total calls offered.
- Excluding hangups in first x seconds within the live rep queue, rather
than including all abandons within the live rep queue.
- Including calls abandoned within IVR/messaging in numerator, rather
than calls abandoned in live rep queue only.
- Including total calls offered to IVR/messaging in denominator, rather
than live rep queue only.
32
AVERAGE CALL LENGTH
PURPOSE--Measure average call length
DEFINITION
- Average time per call that elapsed between rep answer and rep hangup.
Measured in minutes and seconds.
CALCULATION
Total talk time + total hold time (after initial rep response)
Total calls answered in live rep queue
- Total hold time in this measurement refers to hold time after the
caller has reached a rep, i.e. hold time between transfers, etc.
- Minutes should be reported in decimal places. Example 6 minutes and 30
seconds should be reported as "6.50".
- Should be system generated via ACD software or a consistent process
that is auditable
- This should be calculated at the lowest level of available detail in
regards to campaign, program, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Reporting minimum time elapsed
- Reporting maximum time elapsed
- Not including all queues relevant to program
- Excluding customer service queue from reporting
- Counting talk time only, rather than including talk time + hold time
after rep response
AVERAGE HOLD TIME
33
PURPOSE--Measure average hold time
DEFINITION
- Average time per call that elapsed between entering the live rep queue
and rep answer. Measured in minutes and seconds.
CALCULATION
Total time between the live rep queue and rep answer for all calls answered in
the live rep queue
Total calls answered in live rep queue
- This does not measure hold time related to IVRs or other messaging
systems that does not result in talking to a live rep.
- Minutes should be reported in decimal places. Example 6 minutes and 30
seconds should be reported as "6.50".
- Should be system generated via ACD software or a consistent process
that is auditable
- This should be calculated at the lowest level of available detail in
regards to campaign, program, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Reporting minimum time elapsed
- Reporting maximum time elapsed
- Not including all queues relevant to program
- Excluding customer service queue from reporting
- Counting talk time only
34
MAIL/FAX PROCESSING RATE
PURPOSE-- Measure response time to mail pieces and faxes
DEFINITION
- The percentage of mail pieces and faxes requiring manual order entry
that are received workday 1 and processed workday y by 12:00 midnight
in relation to total mail pieces and faxes received workday 1 by 12:00
midnight
CALCULATION
Mail pieces + faxes received workday 1 and processed workday y by 12:00 midnight
Total mail pieces + faxes received workday 1 by 12:00 midnight
- Mail and Fax pieces received workday 1 and processed workday 1 by
12:00 midnight count as same day. Mail and Fax pieces received workday
1 and processed workday 2 by 12:00 midnight count as 24 hours or one
day.
- This measurement includes complaint and not-compliant orders.
- Processed = order entry, non-compliance procedure enacted, or order
confirmation sent.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
- Mail pieces include activation cards or equivalent information for the
MSDN and TechNet programs.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x% in y days (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Not date stamping/recording all pieces with date they arrived via
mail, fax or download and allowing them to pile up unstamped.
- Not capturing date received in order entry system. Not capturing
non-compliant orders in calculation.
- Counting backlog in numerator for subsequent days measurement.
- Mistaking requirement of weekly performance reporting for a service
level of 5 days, rather than reporting a weekly performance to
contractual standard of y days.
- Averaging daily performance, rather than re-summing all mail and fax
received workday 1 and processed workday y by 12:00 midnight in
numerator and re-summing total mail and fax received for the week in
denominator.
35
APPLICATION PROCESSING RATE
PURPOSE-- Measure response time to applications
DEFINITION
- The percentage of applications received by download, web, mail, or fax
requiring manual order entry that are received workday 1 and processed
workday y by 12:00 midnight in relation to total mail pieces and faxes
received workday 1 by 12:00 midnight
CALCULATION
Applications received workday 1 and processed workday y by 12:00 midnight
Total mail pieces + faxes received workday 1 by 12:00 midnight
- Applications received workday 1 and processed workday 1 by 12:00
midnight count as same day. Mail and Fax pieces received workday 1 and
processed workday 2 by 12:00 midnight count as 24 hours or 1 day.
- Processed = order entry or non-compliance procedure enacted.
- This measurement includes both compliant and non-compliant
applications.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x% in y days (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Not date stamping all applications with the date they arrived via
mail, fax, or download and allowing them to pile up unstamped.
- Not capturing date received in application entry system. Not capturing
non-compliant applications in calculation.
- Counting backlog in numerator for subsequent days measurement.
- Mistaking requirement of weekly performance reporting for a service
level of 5 days, rather than reporting a weekly performance to
contractual standard of y days.
- Averaging daily performance, rather than re-summing all applications
received workday 1 and processed workday y by 12:00 midnight in
numerator and re-summing total applications received for the week in
denominator.
36
COMPLIANT ORDER PERCENTAGE
PURPOSE-- Measure the percentage of compliant orders received
DEFINITION
- The percentage of compliant orders received in relation to the total
number of orders received.
CALCULATION
Number of Compliant Orders Received
Total Number of Orders Received
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Measuring orders that did not include valid payment, credit card
approval, etc.
- Including orders where the customer had to be contacted to complete
the order information.
37
MANUAL PROCESSING RATE ON WEB ORDERS
PURPOSE-- Measure clerical response time to manual order entry/processing of web
orders
DEFINITION
- The percentage of web orders requiring manual processing that were
submitted by enduser workday 1 and manually processed workday y by
12:00 midnight in relation to total web orders requiring manual
processing submitted by enduser workday 1 by 12:00 midnight
CALCULATION
Web Orders submitted workday 1 and manually processed workday y by 12:00
midnight
Total web orders submitted workday 1 by 12:00 midnight
- Web orders submitted workday 1 and processed workday 1 by 12:00
midnight count as same day. Web orders submitted workday 1 and
processed workday 2 by 12:00 midnight count as 24 hours or 1 day.
- Processed = order entry or non-compliance procedure enacted.
- This measurement includes compliant and non-compliant orders.
- Submitted by enduser = ms.com submit date
- This should be calculated at the lowest level of available detail in
regards to web site and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x% in y days (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Measuring against date received by vendor, rather than date submitted
by enduser.
- Measuring against order entry date, rather than date submitted by
enduser.
- Not capturing non-compliant orders in calculation.
- Counting backlog over y days as processed in numerator for subsequent
days measurement.
- Mistaking requirement of weekly performance reporting for a
measurement of 5 days, rather reporting a weekly performance to
contractual standard of y days.
- Averaging daily performance, rather than re-summing all web orders
received workday 1 and processed workday y by 12:00 midnight in
numerator and re-summing total web orders received for the week in
denominator.
- Excluding Saturday and Sunday from calculation.
38
FREE FLOW EMAIL PROCESSING RATE
PURPOSE-- Measure rep response time to free flow emails
DEFINITION
- The percentage of email requests received workday 1 and responded to
workday y by 12:00 midnight in relation to total emails received
workday 1 by 12:00 midnight
CALCULATION
Email requests received workday 1 and responded to workday y by 12:00 midnight
Total email requests received workday 1 by 12:00 midnight
- Responded to = Order entry, non-compliance procedures enacted, or
response to inquiry sent. Does not include auto reply.
- Excludes eforms received by vendors
- Email requests received workday 1 and resolved workday 1 by 12:00
midnight count as same day. New email requests received workday 1 and
resolved workday 2 by 12:00 midnight count as 24 hours or one day.
- This should be calculated at the lowest level of available detail in
regards program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x% in y days (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Measuring receipt as date of first processing, rather than date
received in inbox
- Counting backlog over y days as resolved in numerator for subsequent
days measurement.
- Averaging daily performance, rather than re-summing all new email
requests received workday 1 and resolved workday y by 12:00 midnight
in numerator and re-summing total new email requests received for the
week in denominator.
39
EFORM PROCESSING RATE
PURPOSE-- Measure rep response time to eforms
DEFINITION
- The percentage of eforms received workday 1 and processed workday y by
12:00 midnight in relation to total eforms received workday 1 by 12:00
midnight
CALCULATION
Eforms received workday 1 and processed workday y by 12:00 midnight
Total eforms received workday 1 by 12:00 midnight
- Processed = Order entry, non-compliance procedures enacted, or
response to inquiry sent. Does not include auto reply.
- Excludes free flow emails
- Eforms received workday 1 and processed workday 1 by 12:00 midnight
count as same day. Eforms received workday 1 and processed workday 2
by 12:00 midnight count as 24 hours or one day.
- This should be calculated at the lowest level of available detail in
regards program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x% in y days (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Measuring receipt as date of first processing, rather than date
received in inbox
- Counting backlog over y days as resolved in numerator for subsequent
days measurement.
- Averaging daily performance, rather than re-summing all eforms
received workday 1 and resolved workday y by 12:00 midnight in
numerator and re-summing total eforms received for the week in
denominator.
40
SUPPORTING METRICS - FRONT END VOLUMES
41
CALLS ANSWERED - LIVE REP IN IVR ENVIRONMENT
PURPOSE-- To segregate volume of calls answered by a live rep only, when both
live rep and IVR exist.
DEFINITION
- Inbound calls answered by a live rep, either initially or as a
transfer from the IVR.
CALCULATION
Count of inbound calls answered by a live rep
- If phone queue is purely live rep and no IVR, this metric should not
be reported, default to Total Calls Answered metric which is
categorized as a Top KPI.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting calls offered rather than calls answered
- Including outbound calls, rather than inbound only
- Double counting internal transfers/escalations
- Including IVR calls completed
42
CALLS ANSWERED - IVR
PURPOSE-- Measure volume of inbound calls answered by an IVR.
DEFINITION
- Inbound calls completed in an IVR
CALCULATION
Count of inbound calls completed in an IVR
- Calls completed in an IVR represent calls where the customer chose at
least one option in the IVR routing system. This does not include
calls that were abandoned during the first routing message. For voice
mail systems that do not support multiple routing options a call that
generates a message is considered "answered"; a call that does not is
abandoned and not counted in this metric.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting calls offered rather than calls answered
- Including outbound calls, rather than inbound only
- Double counting internal transfers/escalations
- Counting calls that were abandoned during or before the first routing
message in an IVR
43
OUTBOUND CALLS
PURPOSE-- Measure volume of outbound calls.
DEFINITION
- Outbound calls made for Microsoft.
CALCULATION
Count of outbound calls made for Microsoft
- This measures the number contacts and does not include dials that did
not result in a completed call script.
- This represents only calls made by the request of Microsoft and should
not include calls made for personal or vendor related reasons.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting calls dialed rather than calls completed
- Including inbound calls, rather than outbound only
44
AVERAGE CALLS PER DAY
PURPOSE-- Measure volume of inbound calls, specifically for fiscal months that
vary in length
DEFINITION
- Average number inbound calls per day answered by a live rep and
inbound calls completed in an IVR
CALCULATION
Count of inbound calls answered by a live rep + inbound calls completed in an
IVR
Sum of days in the reporting period
- If an IVR is not used, then this metric becomes purely a count of
average inbound calls answered by a live rep.
- Calls completed in an IVR represent calls where the customer chose at
least one option in the IVR routing system. This does not include
calls that were abandoned during the first routing message. For voice
mail systems that do not support multiple routing options a call that
generates a message is considered "answered"; a call that does not is
abandoned and not counted in this metric.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting calls offered rather than calls answered
- Including outbound calls, rather than inbound only
- Double counting internal transfers/escalations
- Excluding IVR calls completed
- Counting calls that were abandoned during or before the first routing
message in an IVR
45
FREE FLOW INBOUND EMAILS RECEIVED
PURPOSE-- Measure volume of inbound free flow emails received.
DEFINITION
- Quantity of incoming orders and inquiries received via free flow email
CALCULATION
Count of incoming email orders received + Count of incoming email inquiries
received
- Each incoming email receipt should be counted separately. Incoming
forward, replies and emails regarding the same subject should be
counted uniquely.
- Should be system generated via email software
- Excludes eforms
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting email orders only, and excluding inquiries
- Counting multiple email receipts from the same customer/on the same
subject as one rather than counting each receipt individually
- Including outbound emails, rather than only counting inbound emails
- Including eforms
46
INBOUND EFORMS RECEIVED
PURPOSE-- Measure volume of inbound eforms
DEFINITION
- Quantity of incoming orders and inquiries received via eform
CALCULATION
Count of incoming eform orders received + Count of incoming eform inquiries
received
- Each incoming eform receipt should be counted separately. Incoming
forward, replies and eforms regarding the same subject should be
counted uniquely.
- Should be system generated via email software
- Excludes free flow email
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting eform orders only, and other eform communication
- Counting multiple eform receipts from the same customer/on the same
subject as one rather than counting each receipt individually
- Including outbound emails or eforms, rather than only counting inbound
eforms
- Including inbound or outbound free flow email
47
FREE FLOW OUTBOUND EMAILS SENT
PURPOSE-- Measure volume of free flow outbound emails sent
DEFINITION
- Quantity of total free flow outbound emails sent including all emails
sent to customers, internal vendor escalation, internal vendor
inquiry, MS escalation, MS inquiry, forwards to other sources, etc.
including those in response to eforms.
CALCULATION
Count of total free flow outbound emails sent
- Each outbound email sent should be counted separately. Outbound
forward, replies and emails regarding the same subject should be
counted uniquely.
- Excludes push-back emails
- Should be system generated via email software
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting outbound emails sent to customers only, and excluding
internal/MS inquiries or escalations
- Counting multiple emails sent on the same subject as one rather than
counting each email sent individually
- Including incoming emails, rather than only counting outbound emails
sent
- Counting replies/forwards only, rather than counting emails initiated
by vendor
- Including push-back emails
48
APPLICATIONS RECEIVED
PURPOSE-- Measure volume of applications received.
DEFINITION
- Quantity of applications received during the reporting period.
CALCULATION
Sum of all applications stamped or matched to existing members' records.
- An application is a request for enrollment or renewal in a specific
Microsoft program.
- Stamped applications are for new enrollment that do not have a
matching record in the application system.
- Matched applications are requests for renewal that have been paired
with the member's information already in the application system.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Only reporting approved applications.
- Excluding applications that are not reconciled.
49
SUPPORTING METRICS - EXAM PERFORMANCE
50
EXAM SERVICE LEVEL
PURPOSE--Measure the timeliness of exams delivered to testing centers.
DEFINITION
- Exams delivered on time from exam service providers to testing
centers.
CALCULATION
Number of correct exams delivered on or before scheduled registration time
Total number of exams scheduled
- The scheduled registration time is the time that the customer has
agreed to take the exam.
- Correct exams represent the same exam that the customer registered to
take at that time.
FREQUENCY--MS Fiscal Month
BENCHMARK - 98% of all exams delivered on or before scheduled registration.
MEASUREMENT TRAPS
- Counting exams delivered in the denominator instead of exams scheduled
- Including incorrect exams in the numerator
51
EXAM DATA SENT - 72 HOURS
PURPOSE--Measure the timeliness of exam data returned to Microsoft.
DEFINITION
- Exam data returned to Microsoft within 72 hours in relation to the
total number of exams delivered.
CALCULATION
Number of delivered exams for which the data was successfully returned
to Microsoft within 72 hours
Total number of exams delivered
- The time that the exam data is returned to Microsoft is measured by
the date of the vendors' file transfer.
- If the file transfer was not successful because of an error on the
part of the vendor, the affected data should not be included in the
numerator.
FREQUENCY--MS Fiscal Month
BENCHMARK - 90%
MEASUREMENT TRAPS
- Counting exams scheduled in the denominator instead of exams delivered
- Including exam data that was not received successfully by Microsoft in
the numerator.
52
EXAM DATA SENT - 120 HOURS
PURPOSE--Measure the timeliness of exam data returned to Microsoft.
DEFINITION
- Exam data returned to Microsoft within 120 hours in relation to the
total number of exams delivered.
CALCULATION
Number of delivered exams for which the data was successfully returned
to Microsoft within 120 hours
Total number of exams delivered
- The time that the exam data is returned to Microsoft is measured by
the date of the vendors' file transfer.
- If the file transfer was not successful because of an error on the
part of the vendor, the affected data should not be included in the
numerator.
FREQUENCY--MS Fiscal Month
BENCHMARK - 100%
MEASUREMENT TRAPS
- Counting exams scheduled in the denominator instead of exams delivered
- Including exam data that was not received successfully by Microsoft in
the numerator.
53
SUPPORTING METRICS - EXAM VOLUMES
54
EXAMS DELIVERED
PURPOSE-- Measure volume of exams delivered
DEFINITION
- Exams delivered to a test center
CALCULATION
Count of exams delivered to a testing center and available for candidates
- This does not include resending the correct exam when the first exam
made available to the candidate was the incorrect.
- This does not include a delivered exam when the candidate was not
present to take the exam.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting exams that were delivered to the testing center but not
available for the candidate, i.e. systems issues, etc.
- Counting exams that were not actually taken by a candidate.
55
EXAMS REGISTERED BY PHONE
PURPOSE-- Measure volume of exams registrations made by phone
DEFINITION
- Exams registrations made by phone
CALCULATION
Count of exams registrations taken by phone and recorded/entered into a
registration system
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting registrations that were made by a media other than phone.
56
EXAMS REGISTERED BY INTERNET
PURPOSE-- Measure volume of exams registrations made over the internet
DEFINITION
- Exams registrations made over the internet
CALCULATION
Count of exams registrations taken from the internet and recorded/entered into a
registration system
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting registrations that were made by a media other than the
internet.
57
EXAMS REGISTERED ON-SITE
PURPOSE-- Measure volume of exams registrations made at a testing center
DEFINITION
- Exams registrations made at a testing center
CALCULATION
Count of exams registrations taken at a testing center and recorded/entered into
a registration system
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting registrations that were not made at a testing center.
58
TOTAL EXAMS REGISTERED
PURPOSE-- Measure volume of exams registrations
DEFINITION
- Exams registrations made at a testing center, by phone, and from the
internet
CALCULATION
Count of exams registrations taken at a testing center, by phone, and from the
internet and recorded/entered into a registration system
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Excluding exam registrations take from one media.
59
SUPPORTING METRICS -FULFILLMENT PERFORMANCE
60
ORDER FILL RATE ONE-OFF OR WELCOME KIT
PURPOSE-- Measure WPO's ability to accept orders and fulfill product in a timely
manner per specific business requirements, measured specific to one-offs and
welcome kits
DEFINITION
- Total valid compliant order lines entered workday 0 by 12:00 midnight
and shipped complete (picked up by carrier) workday y by 12:00
midnight in relation to total valid compliant order lines entered
workday 0 by 12:00 midnight.
CALCULATION
Number or compliant order lines received workday 0 and
shipped within specified y time period
Total compliant order lines received workday 0
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- Order lines received workday 0 and shipped workday 0 = same day
- Order lines received workday 0 and shipped workday 1 = 1 day
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x % in y days. (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Excluding order lines that were not shipped because the product was
backordered, i.e. poor forecasting or problems with a supplier
- Measuring workday 0 from the packslip print instead of actual order
entry
- Measuring workday 0 from the third-party feed to fulfillment center,
rather than capturing actual front-end order date
61
ORDER FILL RATE SUBSCRIPTION/MONTHLY MAILER
PURPOSE -- Measure WPO's ability to accept orders and fulfill product in a
timely manner per specific business requirements. Measured specific to
subscription monthly mailers, such as autoships for Courseware, Interim &
Quarterlies for MSDN, Monthly MSDN and TechNet, and Update cycle for Select.
DEFINITION
- Total valid compliant order lines shipped starting scheduled ship day
1 through scheduled ship day y in relation to total valid compliant
order lines entered from cycle pull.
CALCULATION
Total compliant order lines shipped through scheduled ship day y
Total compliant order lines received from cycle pull
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- Order lines shipped workday 1 = 1 day
- Order lines shipped workday 2 = 2 days
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x % in y days. (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Starting timing from day of cycle pull, rather than ship day 1.
- Excluding order lines that were not shipped because the product was
backordered, i.e. poor forecasting or problems with a supplier
62
ORDER FILL RATE EXPEDITE/RUSH ORDERS
PURPOSE-- Measure WPO's ability to accept orders and fulfill product in a timely
manner per specific business requirements, measured specific to expedite and
rush orders.
DEFINITION
- Total valid compliant order lines entered workday 0 by 12:00 midnight
and shipped complete (picked up by carrier) workday y by 12:00
midnight in relation to total valid compliant order lines entered
workday 0 by 12:00 midnight.
CALCULATION
Total compliant order lines received workday 0 and shipped
within specified y time period
Total compliant order lines received workday 0
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- Order lines received workday 0 and shipped workday 0 = same day
- Order lines received workday 0 and shipped workday 1 = 1 day
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x % in y days. (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Excluding order lines that were not shipped because the product was
backordered, i.e. poor forecasting or problems with a supplier
- Measuring workday 0 from the packslip print instead of actual order
entry
- Measuring workday 0 from the third-party feed to fulfillment center,
rather than capturing actual front-end order date
63
ORDER FILL RATE REPLACEMENT PARTS/RESHIPS
PURPOSE-- Measure WPO's ability to accept orders and fulfill product in a timely
manner per specific business requirements, measured specific to replacement
parts and reships.
DEFINITION
- Total valid compliant order lines entered workday 0 by 12:00 midnight
and shipped complete (picked up by carrier) workday y by 12:00
midnight in relation to total valid compliant order lines entered
workday 0 by 12:00 midnight.
CALCULATION
Total compliant order lines received workday 0 and shipped
within specified y time period
Total compliant order lines received workday 0
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- Order lines received workday 0 and shipped workday 0 = same day
- Order lines received workday 0 and shipped workday 1 = 1 day
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x % in y days. (see statement of work or Microsoft Vendor Account
Manager)
MEASUREMENT TRAPS
- Excluding order lines that were not shipped because the product was
backordered, i.e. poor forecasting or problems with a supplier
- Measuring workday 0 from the packslip print instead of actual order
entry
- Measuring workday 0 from the third-party feed to fulfillment center,
rather than capturing actual front-end order date
64
ORDER FILL RATE - MS DICTATED HOLDS
PURPOSE-- Measure WPO's ability to accept orders and fulfill product in a timely
manner per specific business requirements, measured specific to product that
Microsoft requests a hold to consolidate or hold to release.
DEFINITION
- Total valid compliant order lines entered workday 0 by 12:00 midnight
and shipped complete (picked up by carrier) workday y by 12:00
midnight in relation to total valid compliant order lines entered
workday 0 by 12:00 midnight.
CALCULATION
Number or compliant order lines received workday 0 and shipped
within specified y time period
Total compliant order lines received workday 0
- The specified y time period in this case is dictated by Microsoft for
a specific purpose. For example, Microsoft may ask the fulfillment
vendor to hold all shipments to a certain region until Thursday of
each week to gain a distribution cost advantage. In this case,
Thursday becomes the specified time period and all order lines that
were received since the last shipment and shipped on the following
Thursday are counted in the numerator above.
- Compliant order lines are those that meet all of the necessary
requirements for fulfillment. They include all of the needed
information, approved payment, and are for a product that has been
released to manufacturing. Order lines for new product are not
compliant until the initial build for that product is complete.
- Order lines that include all of the required information but are for
product that is backordered because of poor forecasting or supplier
problems ARE COMPLIANT.
- Order lines received workday 0 and shipped workday 0 = same day
- Order lines received workday 0 and shipped workday 1 = 1 day
- This should be calculated at the lowest level of available detail in
regards to program, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - x % in y days. (see Microsoft Vendor Account Manager and calculation
notes above)
MEASUREMENT TRAPS
- Excluding order lines that were not shipped because the product was
backordered, i.e. poor forecasting or problems with a supplier
- Measuring workday 0 from the packslip print instead of actual order
entry
65
- Measuring workday 0 from the third-party feed to fulfillment center,
rather than capturing actual front-end order date
66
LONGEST TIME TO ORDER FILL (TAIL)
PURPOSE-- Measure the longest time, in days, to fulfill every order.
DEFINITION
- Longest time period from each day's order entry to that same day's
order lines shipping 100% complete.
CALCULATION
(Maximum time between order entry and shipment for any order line) -
(Order Fill Rate target, commonly referred to as "y", for
the respective product)
EXAMPLE: 100 Welcome Kit orders entered day 1. 95 fulfilled within 2 days; order
fill rate target. Last order line fulfilled on the day 5. Longest Time to Order
Fill = 3. 10,000 subscription orders entered day 1. 9900 subscription orders
fulfilled by day 5; order fill rate target. Last subscription order fulfilled on
day 11. Longest Time to Order Fill = 6. If both examples ship in the same
reporting period, Longest Time to Order Fill to be reported is 6.
- Order lines received workday 0 and shipped workday 0 = same day.
- Order lines received workday 0 and shipped workday 1 = 1 day
- If the fulfillment of this order line bridges two or more reporting
periods, its total time to fulfill should be included in the reporting
period that it was shipped.
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - 1 Day
MEASUREMENT TRAPS
- Not measuring this metric on single order lines.
- Not reporting an order line that's time to fulfill bridged two or more
reporting periods.
- Measuring order entry from the packslip print instead of actual order
entry
- Measuring order entry from the third-party feed to fulfillment center,
rather than capturing actual front-end order date
- Not subtracting the order fill rate target time period.
67
INVENTORY TURNS
PURPOSE -- Measure how often the inventory "turns over" or how many times one
month's inventory is sold in a year.
DEFINITION
- The last three months' unit sales annualized in relation to the
current month's ending inventory in units.
CALCULATION
(The last three months' sales in units * 4)
The current month's ending inventory in units
- If this was to be calculated for April, it would be the number units
sold in February, March, and April multiplied by four and divided by
April's ending inventory in units.
FREQUENCY--MS Fiscal Month
BENCHMARK - x days (see statement of work or Microsoft Vendor Account Manager).
MEASUREMENT TRAPS
- Not including the current month in the three months' unit sales.
68
INVENTORY ACCURACY
PURPOSE -- Measure the accuracy of inventory tracking systems and controls.
DEFINITION
- The number of units adjusted as the result of a physical inventory or
cycle count in relation to the total number of units counted.
CALCULATION
1 - (the gross number of units adjusted / the gross number of units counted)
- This is calculated from the results of a physical inventory or cycle
count.
- The "gross number of units adjusted" represents the adjustment in
units made to the books or inventory tracking system in order for it
to match the number of units that were physically counted.
- The "gross number of units counted" represents the total number of
units counted during the count.
FREQUENCY--MS Fiscal Month
BENCHMARK - 99%
MEASUREMENT TRAPS
- Using data from an invalid physical inventory or cycle count.
69
SUPPORTING METRICS -RELEASE PERFORMANCE
70
POST HANDOFF/RTM ECNS
PURPOSE-- Measure Microsoft changes to BOMs and SKUs after scheduled/agreed upon
frozen date.
DEFINITION
- Number of changes to the BOM after BOM final and handed off to vendor.
CALCULATION
Number of BOM changes
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK -zero
MEASUREMENT TRAPS
- Not counting multiple BOMs affected by one change
- Not counting multiple changes that affect one BOM
71
PROOF CYCLE TIME (COURSEWARE)
PURPOSE-- Measure vendor's ability to process proof and return to Microsoft
DEFINITION
- Number of days for proof to be created and returned to MS for
approval.
CALCULATION
Number of days
- MS handoff to vendor is day 1, receipt by MS is 1 day. Excluding
non-workdays.
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK - Courseware: 3 days
MEASUREMENT TRAPS
- Excluding the handoff day
- Excluding the receipt day
- Including non-workdays
72
LONGEST DELTA TO ORDERABILITY (COURSEWARE)
PURPOSE -- Measure Microsoft's ability to hand-off good product the first time.
DEFINITION
- Number of days of longest cycle time to orderability.
CALCULATION
Number of days
- MS hand off is day 1, MS approval for orderability is 1 day. Excluding
non- workdays.
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK - Courseware: 10 days
MEASUREMENT TRAPS
- Excluding the handoff day
- Excluding the receipt day
- Including non-workdays
73
SUPPORTING METRICS -MANUFACTURING PERFORMANCE
74
SHIPPED COMPLETE X DAYS LATE
PURPOSE-- Measure vendor's ability to ship within specified time frames
DEFINITION
- Number of days beyond the scheduled/agreed upon time for all shipments
to be complete for a given subscription cycle run.
CALCULATION
Number of days
- If scheduled/agreed for Sep 3 and delivered on Sep 4 - result would be
1.
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK - zero
MEASUREMENT TRAPS
- Including the due date in calculation
- Excluding non-workdays
75
MANUFACTURING CYCLE TIME -SUBSCRIPTION/MONTHLY MAILER
PURPOSE-- Measure the time to manufacture MS product
DEFINITION
- Total number of days from RTM (workday 1) to available to ship for all
items
- Weighted Average of Subscription Events/Monthly Mailers. Includes
Subscription events, monthly mailers, and inventory builds for
Courseware.
CALCULATION
Number of days
- RTM counts as 1 day, and date available to ship counts as 1 day.
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK - x days (see statement of work or Microsoft Vendor Account Manager).
MEASUREMENT TRAPS
- Excluding RTM day
- Excluding available to ship day
76
MANUFACTURING CYCLE TIME -ONE-OFF OR WELCOME KIT
PURPOSE-- Measure the time to manufacture MS product
DEFINITION
- Total number of days from RTM/Receipt of Order (workday 1) to
available to ship for all items (Welcome Kits, daily orders for
Courseware, demand print for Courseware, etc).
CALCULATION
Number of days
- RTM/Receipt of Order counts as 1 day, and date available to ship
counts as 1 day.
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK - x days (see statement of work or Microsoft Vendor Account Manager).
MEASUREMENT TRAPS
- Excluding RTM day
- Excluding available to ship day
77
SUPPORTING METRICS - MANUFACTURING VOLUMES
78
UNITS SHIPPED FROM A MANUFACTURING SITE
PURPOSE-- Measure volume of units shipped from a manufacturing site.
DEFINITION
- Total quantity of confirmed End Items shipped from the manufacturing
site.
CALCULATION
Sum of shipped quantities by end item SKU that are shipped
from a manufacturing site
- Manufacturing site represents the facility where the end-item was
assembled complete.
- End Items are measured by Manufacturing End Item SKU.
- Includes shipments to regional vendor distribution facilities.
- A reship counts as 1 end item SKU shipped
- This should be calculated at the lowest level of available detail in
regards to campaign, country, and MS region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK -n/a
MEASUREMENT TRAPS
- Reporting by SKU, rather than summary of all SKUs
- Summing at a level different than end item SKU
- Excluding reships, comped shipments, replacement parts, supplemental
parts, marketing parts from count
- Reporting units at a program level and not broken out by campaign
- Excluding shipments to regional vendor distribution facilities
79
PRINT IMPRESSIONS
PURPOSE-- Measure print volume
DEFINITION
- Total number of print impressions manufactured or purchased
CALCULATION
Count of print impressions manufactured or purchased
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK -n/a
MEASUREMENT TRAPS
- Excluding print impressions purchased
80
CDS REPLICATED/PURCHASED
PURPOSE-- MEASURE CD VOLUME
DEFINITION
- Total number of CDs replicated or purchased
CALCULATION
Count of CDs replicated or purchased
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK -n/a
MEASUREMENT TRAPS
- Excluding CDs purchased
81
SUPPORTING METRICS - LOGISTICS PERFORMANCE/VOLUMES
82
CARRIER PERFORMANCE
PURPOSE-- Measure carrier performance relative to the carrier's quoted
distribution time
DEFINITION
- Number of shipments delivered to the consignee within the quoted
distribution time in relation to the total number of carrier shipments
due; recognizes standard allowances for international gateway and
customs delays outside the carrier's influence.
CALCULATION
Number of carrier shipments due to the consignee during the reporting period
and delivered to the consignee within the quoted distribution time
Total number of carrier shipments due to the consignee during the reporting
period
- Consignee - "ship to" party as reflected on the airway bill. Consignee
could be customs, Microsoft subsidiary, disti, intermediate consignee,
freight forwarder, or enduser.
- See Carrier Shipments - Due definition page
- This should be calculated at the lowest level of available detail in
regards to program and MS region.
FREQUENCY--MS Fiscal Month
BENCHMARK - 95% of shipments delivered to the consignee within the quoted
distribution time
MEASUREMENT TRAPS
- Measuring delivery to alternate consignee rather than delivery to the
consignee as reflected on the airway bill.
- Measuring shipments picked-up or delivered during a reporting period
rather than those due during the reporting period.
- Including conditions beyond the carrier's control, i.e. bad addresses,
customs holds, consignee not at home, customer/consignee refused, etc.
83
CARRIER SHIPMENTS - PICKED UP
PURPOSE-- Measure the volume of shipments picked-up by the ship carrier
DEFINITION
- - Number of shipments picked up by the ship carrier during the reporting
period.
CALCULATION
Count of shipments picked up by the ship carrier
- - Includes shipments to a single consignee and bulk shipments to Microsoft
subsidiaries, distis, etc.
- - Shipment - Single airway bill
- - This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting end item SKUs, rather than shipments as measured by airway
bills.
- Counting shipments due, rather than shipments picked up.
- Counting shipments delivered, rather than shipments picked up.
84
SUPPORTING METRICS - EUROPE AND AMME SPECIFIC PERFORMANCE/VOLUMES
85
CUSTOMER QUERY RESPONSE RATE
PURPOSE--Measure response time to queries raised by customers
DEFINITION
- The percentage of customer queries responded to within 24 hours in
relation to the total number of queries received.
CALCULATION
Number of customer queries responded to within 24 hours of receipt
Total number of queries received
- Customer Query Response Rate should be reported based on due date.
Although customer queries may have been responded to prior to the
reporting period in which they were due, they should be measured and
reported based on due date.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - 95% / 24 hours
MEASUREMENT TRAPS
86
REPORTS SUBMITTED ON-TIME RATE
PURPOSE-- Measure quality and timeliness of reports submitted by vendors
DEFINITION
- The number of reports submitted correct and on-time to Microsoft EOC
or another Microsoft Vendor in relation to the total number of reports
due.
CALCULATION
Number of reports submitted correct and on-time to Microsoft EOC or another
MS Vendor
Total number of reports due for submission
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - 95% / 24 hours
MEASUREMENT TRAPS
- Excluding reports to other vendors.
- Reviewing only timeliness and not the quality of reports.
87
SHIPPING ISSUES AS % OF CARRIER SHIPMENTS - DUE
PURPOSE--Measure shipping issues in relation to carrier shipments - due.
DEFINITION
- The number of shipping issues by country received from customers in
relation to carrier shipments - due.
CALCULATION
Number of shipping issues received from customers
Total number of carrier shipments - due
- Required by country and by program
- Shipping Issues include;
- Product not delivered
- Late delivery
- Incorrect address
- Incorrect product delivered
- Damaged product
- Carrier Shipments - Due is defined separately
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - None
MEASUREMENT TRAPS
- Including shipping inquiries in shipping issues
88
NUMBER OF FTES
PURPOSE-- Measure number of Full Time Equivalents answering telephone calls.
DEFINITION
- - Full Time Equivalents answering telephone calls.
CALCULATION
Count of Full Time Equivalents answering telephone calls.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Including data entry employees.
- Omitting staff who spent part of their time answering calls.
89
SUPPORTING METRICS - ECOMMERCE VOLUMES
90
WEB HITS BY REASON CODE
PURPOSE--Measure volume of web hits by reason code on vendor managed site
DEFINITION
- Number of total web hits to site by reason code
CALCULATION
Count of web hits by reason code, where reason code maps to web page
- Should be system generated via internet
- Separate pages for stages in order process
- Separate tracking by program by page
- MS to provide reason codes
- This should be calculated at the lowest level of available detail in
regards to web site, program, and region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting web orders only, rather than total hits
- Aggregating data on multiple sites, rather than reporting hits per
site
- Aggregating multiple program data together, when it can be separated
91
TOTAL WEB HITS
PURPOSE--Measure volume of total web hits on vendor managed site
DEFINITION
- Number of total web hits to site
CALCULATION
Count of total web hits to site = Sum of all web hits by reason code
- Should be system generated via internet
- Separate tracking by program by page
- This should be calculated at the lowest level of available detail in
regards to web site, program, and region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting web orders only, rather than total hits
- Aggregating data on multiple sites, rather than reporting hits per
site
- Aggregating multiple program data together, when it can be separated
92
PUSH-BACK EMAIL ORDER LINES BY REASON CODE
PURPOSE-- Measure volume of push-back email order lines by reason code
DEFINITION
- Number of system generated push-back emails by reason code, as a
multiple of order lines and addressees
CALCULATION
Count of system generated push-back emails sent by reason code;
multiple of email addressee order lines summed to reason code
- Example: Order confirmation for 10 email addressees with 2 order lines
each = count of 20.
- Should be system generated via email software
- MS to supply reason codes, i.e. order confirmation, stock out, order
line shipped
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Not calculating email addressees as a multiple
- Not calculating order lines as a multiple
93
TOTAL PUSH-BACK EMAIL ORDER LINES
PURPOSE-- Measure volume of push-back email order lines
DEFINITION
- Number of system generated push-back emails, as a multiple of order
lines & addressees
CALCULATION
Count of system generated push-back emails as a multiple of email
addressee order lines
- Example: Total push-back emails for 20 email addressees with 4 order
lines each = count of 80.
- Should be system generated via email software
- This should be calculated at the lowest level of available detail in
regards to program and region.
FREQUENCY--MS Fiscal Week and MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Not calculating email addressees as a multiple
- Not calculating order lines as a multiple
94
SUPPORTING METRICS - DEVELOPER METRICS
95
ACTIVE SUBSCRIPTIONS - PAID
PURPOSE-- Measure the number of active subscriptions that were originally paid
for
DEFINITION
- Number of existing/active subscriptions that were paid for when first
activated.
CALCULATION
Count of existing/active subscriptions that were paid for when first activated
- Should be generated from the subscription database
- A new subscription should be recognized in the reporting period in
which the Welcome Kit or similar physical deliverable is shipped to
the customer.
- This measures the number of subscriptions at a point in time (the last
day of the fiscal month) and not year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting subscriptions that were activated as a program benefit or
were given away by marketing.
- Including new subscriptions if the Welcome Kit or similar physical
deliverable has not shipped to the customer.
- Reporting a single month's new paid subscriptions
96
ACTIVE SUBSCRIPTIONS - COMPED BY MARKETING
PURPOSE-- Measure the number of active subscriptions that were given away by
marketing
DEFINITION
- Number of existing/active subscriptions that were originally given
away by marketing, free of charge.
CALCULATION
Count of existing/active subscriptions that were originally given away by
marketing, free of charge
- Should be generated from the subscription database
- A new subscription should be recognized in the reporting period in
which the Welcome Kit or similar physical deliverable is shipped to
the customer.
- This measures the number of subscriptions at a point in time (the last
day of the fiscal month) and not year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting subscriptions that were activated as a program benefit or
paid for in full.
- Including new subscriptions if the Welcome Kit or similar physical
deliverable has not shipped to the customer.
- Reporting a single month's new comped subscriptions
97
ACTIVE SUBSCRIPTIONS - MCP
PURPOSE-- Measure the number of active subscriptions that were activated as a
MCP (Microsoft Certified Professional) benefit
DEFINITION
- Number of existing/active subscriptions that were originally activated
as a benefit of the MCP program
CALCULATION
Count of existing/active subscriptions that were originally activated as a
benefit of the MCP program
- Should be generated from the subscription database
- A new subscription should be recognized in the reporting period in
which the Welcome Kit or similar physical deliverable is shipped to
the customer.
- This measures the number of subscriptions at a point in time (the last
day of the fiscal month) and not year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting subscriptions that were given away by marketing free of
charge or paid for in full.
- Including new subscriptions if the Welcome Kit or similar physical
deliverable has not shipped to the customer.
- Reporting a single month's new MCP subscriptions
98
UPGRADES
PURPOSE-- Measure the number of subscription upgrades
DEFINITION
- Number of existing subscribers that upgrade to a higher level
subscription during a given month
CALCULATION
Count of existing subscriptions that are upgraded to a higher level
during the fiscal month
- Should be generated from the subscription database
- The upgrade should be recognized in the reporting period in which the
Welcome Kit or similar physical deliverable is shipped to the
customer.
- This does not channel upgrades
- The upgrade should be reported with the Campaign Code of the new
subscription level.
- This should be measured for a single month's activity.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting renewals at the same level
- Reporting the upgrade in a period other than that in which the Welcome
Kit or similar physical deliverable was shipped to the customer.
- Reporting year to date upgrades
99
RENEWALS AT THE SAME LEVEL
PURPOSE-- Measure the number of subscription renewals during a given month
DEFINITION
- Number of existing subscriptions that are renewed at the same program
level
CALCULATION
Count of existing subscriptions that are renewed at the same level
during the fiscal month
- Should be generated from the subscription database
- The renewal should be recognized in the reporting period in which the
first physical deliverable related to the new subscription is shipped
to the customer. This physical deliverable could include but is not
limited to a Welcome Kit, Incremental Kit, or Order Confirmation.
- This includes channel renewals and does not include renewals that
remain unpaid.
- This should be measured for a single month's activity.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting upgrades
- Reporting the renewal in the period it was paid for and not the period
the new subscription was shipped to the customer.
- Reporting year to date renewals
100
EXPIRED SUBSCRIPTIONS
PURPOSE-- Measure the number of subscriptions that expired during a given month
DEFINITION
- Number of existing subscriptions that were not renewed or upgraded
upon expiration
CALCULATION
Count of existing subscriptions that were not renewed or upgraded upon
expiration during the fiscal month
- Should be generated from the subscription database
- The expiration should be recognized in the reporting period in which
the subscription was marked as expired.
- This should be measured for a single month's activity.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting upgrades to a higher level
- Reporting the number of year to date expired subscriptions
101
CANCELLED/RETURNED SUBSCRIPTIONS
PURPOSE-- Measure the number of subscriptions that were cancelled or returned
during a given month.
DEFINITION
- Number of existing subscriptions that are cancelled or returned by the
customer
CALCULATION
Count of existing subscriptions that are cancelled or returned during the
fiscal month
- Should be generated from the subscription database
- The return or cancellation should be recognized in the reporting
period in which the subscription's status is changed to "cancelled".
- This should be measured for a single month's activity.
- This does not include undeliverable shipments.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting the returned or cancelled subscription before the refund is
made.
- Reporting the number of year to date cancelled subscriptions
- Including returns from undeliverable shipments
102
COMPED SUBSCRIPTIONS - MARKETING
PURPOSE-- Measure the number of subscription that were given away by marketing
during a given month
DEFINITION
- Number of subscriptions that were given away, free of charge, and not
benefits from other programs
CALCULATION
Count of subscriptions that are given to marketing or shipped to a customer,
free of charge, with marketing's approval during the fiscal month
- Should be generated from the subscription database
- The comped subscription should be recognized in the reporting period
in which the new subscription was shipped to marketing or the
customer.
- This does not include channel activations.
- These are subscriptions that do not generate revenue.
- This should be measured for a single month's activity.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting comped subscriptions that are related to other programs'
benefits
- Reporting year to date comped subscriptions
103
COMPED SUBSCRIPTIONS - MCP
PURPOSE-- Measure the number of subscriptions that shipped as MCP benefits
during a given month
DEFINITION
- Number of subscriptions that are shipped to new MCPs as part of MCP's
program benefits
CALCULATION
Count of subscriptions that are shipped as part of MCP benefits during the
fiscal month
- Should be generated from the subscription database
- MCP - Microsoft Certified Professional
- The comped subscription should be recognized in the reporting period
in which the new subscription was shipped to the customer.
- This does not include channel activations.
- These are subscriptions that do not generate revenue.
- This should be measured for a single month's activity.
- This should be calculated at the lowest level of available detail in
regards to subscription level and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting comped subscriptions that are related to MCSP's benefits or
given away by marketing.
- Reporting year to date comped subscriptions
104
NEW CHANNEL SUBSCRIPTIONS
PURPOSE-- Measure the number of new subscriptions purchased from the reseller
channel during a given month
DEFINITION
- Number of new subscriptions purchased from a reseller
CALCULATION
Count of new subscriptions that were purchased from a reseller during the
fiscal month
- Should be generated from the subscription database
- This does not include upgrades or renewals
- This should be measured for a single month's activity.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting comped subscriptions that are related to another program's
benefits or given away by marketing.
- Counting other paid subscriptions that were not purchased through the
reseller channel.
- Reporting year to date subscriptions
- Including upgrades or renewals
105
NEW DIRECT SUBSCRIPTIONS
PURPOSE-- Measure the number of new subscriptions purchased directly from
Microsoft during a given month
DEFINITION
- Number of new subscriptions purchased directly from Microsoft or its
respective vendor during the fiscal month
CALCULATION
Count of new subscriptions that were purchased directly from Microsoft
or its respective vendor
- This does not include channel activations or comped subscriptions
- This should be measured for a single month's activity.
- Should be generated from the subscription database
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting comped subscriptions that are related to another program's
benefits or given away by marketing.
- Counting paid subscriptions that were purchased through the reseller
channel.
- Reporting year to date subscriptions
106
REMAINING SUBSCRIPTIONS ISSUES (0)
PURPOSE-- Measure the number of active subscriptions that have no remaining
issues
DEFINITION
- Number of active subscriptions that have no remaining issues to go
CALCULATION
Count of active subscriptions that have no remaining issues to go as recorded
on the last day of the fiscal month
- Active subscriptions includes only those eligible to receive interims
- Should be generated from the subscription database
- This measures the number of subscriptions at a point in time and not
year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting any subscriptions that have remaining issues
- Not counting subscriptions that are still eligible for interims
107
REMAINING SUBSCRIPTION ISSUES (1)
PURPOSE-- Measure the number of active subscriptions that have one issue
remaining
DEFINITION
- Number of active subscriptions that have one issue remaining
CALCULATION
Count of active subscriptions that have one issue to go as recorded on the
last day of the fiscal month
- Active subscriptions includes only those eligible to receive interims
- Should be generated from the subscription database
- This measures the number of subscriptions at a point in time and not
year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
Counting any subscriptions that have other than one remaining issue
108
REMAINING SUBSCRIPTION ISSUES (2)
PURPOSE-- Measure the number of active subscriptions that have two issues
remaining
DEFINITION
- Number of active subscriptions that have two issues remaining
CALCULATION
Count of subscriptions that have two issues to go as recorded on the last day of
the fiscal month
- Active subscriptions includes only those eligible to receive interims
- Should be generated from the subscription database
- This measures the number of subscriptions at a point in time and not
year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting any subscriptions that have other than two remaining issues
109
REMAINING SUBSCRIPTION ISSUES (3)
PURPOSE-- Measure the number of active subscriptions that have three issues
remaining
DEFINITION
- Number of active subscriptions that have three issues remaining
CALCULATION
Count of subscriptions that have three issues to go as recorded on the last day
of the fiscal month
- Active subscriptions includes only those eligible to receive interims
- Should be generated from the subscription database
- This measures the number of subscriptions at a point in time and not
year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting any subscriptions that have other than three remaining issues
110
REMAINING SUBSCRIPTION ISSUES (4+)
PURPOSE-- Measure the number of active subscriptions that have four or more
issues remaining
DEFINITION
- Number of active subscriptions that have four or more issues remaining
CALCULATION
Count of subscriptions that have four or more issues to go as recorded on the
last day of the fiscal month
- Active subscriptions includes only those eligible to receive interims
- Should be generated from the subscription database
- This measures the number of subscriptions at a point in time and not
year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting any subscriptions that have less than four remaining issues
111
SUPPORTING VOLUME - BETA CD VOLUMES
112
BETA CD SUBSCRIPTIONS - MARKETING
PURPOSE-- Measure the number of active Beta CD subscriptions that were nominated
DEFINITION
- Number of active Beta CD subscriptions that were nominated and
activated with a signed NDA
CALCULATION
Count of active Beta CD subscriptions that were nominated and activated with a
signed NDA
- Should be generated from the subscription database
- NDA - Non-Disclosure Agreement.
- This measures the number of subscriptions at a point in time and not
year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting any subscriptions that were activated as a MCP benefit
113
BETA CD SUBSCRIPTIONS - MCP
PURPOSE-- Measure the number of active Beta CD subscriptions that were activated
as a MCP (Microsoft Certified Professional) benefit
DEFINITION
- Number of active Beta CD subscriptions that were activated as a MCP
benefit
CALCULATION
Count of active Beta CD subscriptions that were activated as a MCP benefit
- Should be generated from the subscription database
- This measures the number of subscriptions at a point in time and not
year to date.
- This should be calculated at the lowest level of available detail in
regards to subscription level, and country.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Counting any subscriptions that were initiated by marketing
114
SUPPORTING METRICS - MSN VOLUMES
115
UNITS SHIPPED - PHONE
PURPOSE-- Measure the quantity of MSN units that were ordered via the phone and
shipped to an End-Customer.
DEFINITION
- Total quantity of MSN units shipped that were ordered via the phone.
CALCULATION
Sum of MSN units ordered via the phone and shipped to an End-Customer
- End-Customer represents the person or party that placed the order.
- This should be calculated at the lowest level of available detail in
regards to campaign, country and MS region.
FREQUENCY--MS Fiscal Month
BENCHMARK -n/a
MEASUREMENT TRAPS
- Reporting units at a program level and not broken out by campaign
116
UNITS SHIPPED - INTERNET
PURPOSE-- Measure the quantity of MSN units that were ordered via the internet
and shipped to an End-Customer.
DEFINITION
- Total quantity of MSN units shipped that were ordered via the
internet.
CALCULATION
Sum of MSN units ordered via the internet and shipped to an End-Customer
- Internet includes email and web.
- End-Customer represents the person or party that placed the order.
- This should be calculated at the lowest level of available detail in
regards to campaign, country and MS region.
FREQUENCY--MS Fiscal Month
BENCHMARK -n/a
MEASUREMENT TRAPS
- Reporting units at a program level and not broken out by campaign
117
UNITS SHIPPED - DOWNLOAD
PURPOSE-- Measure the quantity of MSN units that were ordered via a download to
the vendor and shipped to an End-Customer.
DEFINITION
- Total quantity of MSN units shipped that were ordered via a download.
CALCULATION
Sum of MSN units ordered via a download and shipped to an End-Customer
- End-Customer represents the person or party that placed the order.
- This should be calculated at the lowest level of available detail in
regards to campaign, country and MS region.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Reporting units at a program level and not broken out by campaign
118
UNITS SHIPPED - MAIL
PURPOSE -- Measure the quantity of MSN units that were ordered via mail or fax
and shipped to an End-Customer.
DEFINITION
- Total quantity of MSN units shipped that were ordered via mail or fax.
CALCULATION
Sum of MSN units ordered via mail or fax and shipped to an End-Customer
- End-Customer represents the person or party that placed the order.
- This should be calculated at the lowest level of available detail in
regards to campaign, country and MS region.
FREQUENCY--MS Fiscal Month
BENCHMARK - n/a
MEASUREMENT TRAPS
- Reporting units at a program level and not broken out by campaign
119
Appendix B to Exhibit A
This MCP Customer Service Calls is attached hereto as Appendix B to the Exhibit
A Statement of Work ("SOW") of the Master Services Agreement entered into by and
between (CHINESE CHARACTERS) Microsoft (China) Co., Ltd. and (CHINESE
CHARACTERS) ATA Testing Authority, Beijing Co., Ltd. ("ATA") on May 16, 2003.
This MCP Customer Service Calls may be updated by Microsoft from time to time
and ATA shall obey the MCP Customer Service Calls in accordance with the Master
Services Agreement and SOW.
The "MCP Customer" provided in this MCP Customer Service Calls shall have the
same meaning as the "Candidate" as defined in the SOW.
MCP CUSTOMER SERVICE CALLS
TABLE OF CONTENTS
MCP CUSTOMER SERVICE CALLS
1.0 OVERVIEW
MCP customers want MCP information as quickly as possible, but without being
"bounced" from person to person. In order to achieve this, representatives
answering these calls must immediately know who is responsible for information
so they can either keep a call or transfer it. This document is intended to make
call responsibility between vendors clear to avoid hesitation and reduce
transfers.
Several vendors involved with the Microsoft Certified Professional program (MCP)
support customer service calls. These vendors are either Regional Educational
Service Centers vendors ("RESCs") or exam testing vendors.
2.0 CALL RESPONSIBILITIES
Call responsibilities are outlined in the table below. Note that in some cases
responsibility overlaps.
2.1 CALL RESPONSIBILITIES GRID
1
3.0 MCP WEB SITE
3.1 MCP WEB SITE TOPICS LIST
Callers should be encouraged to go to the MCP web site at all appropriate
opportunities. The web includes topics such as the ones listed in 3.1, MCP Web
Site Topics List. This list also forms the broad range of topics for which RESCs
are responsible. Note that this list may not be complete and may change at any
time.
MCP Web Site Information
3.2 LIST OF REASONS TO CALL AN RESC
It is also important to know that the public MCP Web Site is the RESCs' major
source of information. This same information is available to customers.
Basically, people call an RESC because they would rather speak to a person than
hunt for information on the web. The only reasons for which customers truly need
to call an RESC are listed below.
2
4.0 REFERRING CALLS
The call referral grids below outline when RESCs should refer customers to an
exam vendor or the web and when exam vendors should refer customers to an RESC
or the web.
4.1 RESC CALL REFERRAL GRID
4.2 EXAM VENDOR CALL REFERRAL GRID
3
Appendix B to Exhibit A
5.0 EXAM REGISTRATION QUESTIONS AND ANSWERS
Below are the most common questions and categories of questions concerning exam
registration, taking exams, external certifications, vouchers and the Microsoft
Sales Specialist program. Note that BOTH exam vendors and RESCs are responsible
for these questions and answers.
5.1 REGISTRATION
Q: Why must a candidate show two forms of ID?
A: Requiring two forms of ID, one with a picture, helps prove that a person is
who they say they are and discourages cheating.
Q: Why is it important to verify candidates' information?
A: If information is not correct, MCPs may not receive their benefits. Also,
it only takes two people a minute to verify information. It can takes
several people a total of more than an hour over a span of many days to fix
incorrect information, plus much annoyance in between.
Q: Why is the e-mail field important?
A: E-mail and Microsoft Web Sites are the primary method that Microsoft uses
to communicate new information to MCPs. Without a valid e-mail address in
the related databases, MCPs may not receive the latest information
regarding new exams, promotions or benefits.
Q: Are P.O. boxes acceptable addresses?
A: No. They are not accepted in North America because most MCP benefits will
not be delivered to a P.O. Box. If a customer absolutely insists, warn them
that they may not receive their benefits.
Q: Can candidates register for an exam twice?
A: Candidates may register for the SAME exam only once at any given time. For
example, they may not be registered for the same exam Monday and again
Friday in case they fail the first time. They may register for two or more
DIFFERENT exams at the same time.
Q: How does a specific promotion work?
A: Information regarding promotions is forwarded before the promotion goes
live. See reference material specific to that promotion.
Q: What is the discount for a specific promotion?
A: Information regarding promotions is forwarded before the promotion goes
live. See reference material specific to that promotion.
Q: Is a candidate eligible for this promotion?
4
Appendix B to Exhibit A
A: Information regarding promotions is forwarded before the promotion goes
live. See reference material specific to that promotion.
Q: What are Upgrade Promotions and who is eligible?
A: When an exam retires and the MCP candidate is required to take the new exam
to keep his/her certification, the candidate can take the exam at 50% off
for a limited time. Refer to WWW.MICROSOFT.COM/EXAMINFO/RETIRED
5.2 EXAM QUESTIONS
Q: How much does it cost to take MCP, Beta, MCT or MSS exams?
A: The exam price is included in tuition through the Ministry of Education
Polytechnic Program. For commercial pricing please visit Microsoft's
website.
Q: What LIVE exams are in development, currently offered, or obsolete?
A: See reference material or Newly released exams at
http://www.microsoft.com/mcp/examinfo/exams.htm Retired exams at
http://www.microsoft.com/mcp/examinfo/retired.htm
Q: What BETA exams are currently offered?
A: See reference material or the certification requirement tables listed for
each certification track. Access from the left-hand column anywhere on the
MCP web site. Example at http://www.microsoft.com/mcp/certstep/mcsein.htm
or use the search tool at
Q: Exam titles and corresponding numbers?
A: See reference material or exam information at
http://www.microsoft.com/mcp/examinfo/exams.htm
Q: Nondisclosure Agreement Issues?
A: In Feb.1998 the MCP Program began requiring candidates to accept the terms
of an NDA before taking an exam. An NDA legally requires candidates to keep
information related to MCP exam items confidential.
See reference material or the following links:
Nondisclosure Agreement text at http://www.microsoft.com/mcp/articles/nda.htm,
Exam security FAQ at http://www.microsoft.com/mcp/certstep/sec_qa.htm, Letter to
MCPs (re exam security) at http://www.microsoft.com/mcp/certstep/letter.htm.
Q: Exam Retake Policies?
A: BETA exams may ONLY be taken once. LIVE exams may be taken a second time at
any time. If candidates fail the second time, they MUST wait at least two
weeks for testing the third time on that exam. This is to protect the
integrity of the exam so people can't memorize the questions and
5
Appendix B to Exhibit A
answers.
See reference material or Exam Security FAQ at
http://www.microsoft.com/mcp/certstep/sec_qa.htm.
Q: Survey Questions?
A: These are general questions regarding the candidates current work
environment (do they work in the IT industry?) as well as questions
regarding how the candidate prepared. Microsoft uses this info to enhance
the MCP exams and study information.
6
Appendix B to Exhibit A
Q: How long does it take to get exam results?
A: Exam results are different for live exams versus beta exams.
1. Results for LIVE exams are given at the end of the exam. Results should
appear in the on-line transcript tool approximately 20 days after the exam
is taken.
2. BETA exam results are generated and mailed to candidates about 8 weeks
after the beta exam closes.
Q: How long is a specific exam (number of questions and time)?
A: Most exams include 50 to 70 items, and the average duration is 90 minutes.
For exams that include adaptive testing the number of items is about 15 to
30; the time required for testing is reduced accordingly.
See reference material or the following links: MCP FAQ at
http://www.microsoft.com/mcp/articles/tesinn.htm, What to expect at the testing
center at http://www.microsoft.com/Train_Cert/mcp/certstep/expect.htm.
Q: What is the passing score for a specific exam?
A: Microsoft DOES NOT provide this information to candidates. Escalate issues
to Microsoft RESC.
Q: In which languages may a candidate take an exam?
A: See reference material.
Q: Exams testing Issues?
A: If a candidate complains about a technical issue with the exam, (example:
an exam shut down half way during the test), it is the exam vendor's
responsibility. Because an RESC will not be able to do anything but listen,
the exam vendor must handle any issues related to a testing center
experience. If the issue is content- or score-related, the candidate should
be referred to an RESC.
Q: Are exams open-book?
A: No.
Q: Can I use a calculator during the exam?
A: Only non-programmable calculators may be brought in for the exam. In
future, all new or republished exams will have built in calculators that
will be activated for exams needing them. Personal calculators are not
allowed for these exams.
Q: What is a Score Report?
7
Appendix B to Exhibit A
A: A score report is a document printed at the end of the exam that states the
candidate's exam score, the exam passing score, and areas for the
candidate's improvement.
Q: Where can candidates go for more information?
A: Refer them to the Web Site at http://www.microsoft.com/mcp/
Q: What is the Exam Experience Like?
A: When you arrive, you will be asked to log in, show two forms of I.D. (one
with a picture) and sign a nondisclosure agreement. Personal items, such as
purses and pagers, are not allowed in the testing area. You can have them
locked in the administrator's desk if you have nowhere else to put them. An
administrator will monitor the exam room and will be available for
assistance. A required demographic survey and an optional testing tutorial
precedes the exam. When you begin the exam, it will tell you how many
questions it contains and how much time is allowed. A clock on the exam
will tell you how many minutes you have left. When you finish all the
questions, you can either review them if you have time left or exit the
exam. When you exit the exam, the administrator will collect any materials,
document your exit time and give you your score report and any personal
items that were locked away.
8
Appendix B to Exhibit A
5.3 EXTERNAL CERTIFICATIONS
Q: How does a candidate get an external certification waiver?
A: To get exams waived, candidates need to fax a copy of their external
certification to their regional exam vendor. The exam vendor enters the
information into the system for transfer to Microsoft. See reference
material. http://www.microsoft.com/mcp/certstep/exempt.htm
5.4 VOUCHERS
Q: What types of vouchers are there?
A: ATA defines the scope of each voucher separately. The terms for one may be
completely different for another. See reference material.
Q: Does a candidate have to test by the voucher expiration date?
A: Yes. See reference material.
Q: How does a candidate redeem the voucher?
A: Candidates give the exam vendor representative the voucher number over the
phone. The representative verifies the number. See reference material.
Q: Can a candidate turn a voucher in for a refund?
A: No. Terms of use are printed on the back of the voucher.
Q: How does a candidate order vouchers?
A: Exam vendors see reference material. RESCs, refer customer to exam vendor.
9
Appendix B to Exhibit A
6.0 SUMMARY OF LINKS
- Exam refund information (5.1.10),
http://www.microsoft.com/Train_Cert/mcp/certstep/examreg.htm
- Exam cost (5.2.1),
http://www.microsoft.com/mcp/faq.htm
- Newly released exams (5.2.2),
http://www.microsoft.com/mcp/examinfo/exams.htm
- Retired exams (5.2.2),
http://www.microsoft.com/mcp/examinfo/retired.htm
- Beta exams currently offered (5.2.3),
http://www.microsoft.com/mcp/certstep/mcsein.htm
- Exam titles and corresponding numbers (5.2.4),
http://www.microsoft.com/mcp/examinfo/exams.htm
- Nondisclosure Agreement text (5.2.5)
http://www.microsoft.com/mcp/articles/nda.htm,
- Exam security FAQ (5.2.5)(5.2.6)
http://www.microsoft.com/mcp/certstep/sec_qa.htm,
- Letter to MCPs (re exam security) (5.2.5),
http://www.microsoft.com/mcp/certstep/letter.htm.
- Exam length (5.2.10)
http://www.microsoft.com/mcp/articles/tesinn.htm
- What to expect at the testing center (5.2.10)
http://www.microsoft.com/Train_Cert/mcp/certstep/expect.htm.
- Where candidates can go for more information (5.2.18)
http://www.microsoft.com/mcp/
http://www.microsoft.com/skills2000
- External certification waivers (5.3.1)
http://www.microsoft.com/mcp/certstep/exempt.htm
- Microsoft Sales Specialist program (5.5.1)
http://www.microsoft.com/salesspecialist/
10
Appendix B to Exhibit A
11
APPENDIX C TO EXHIBIT A
This Missing Exam Information and Data Feed Processes is attached hereto as
Appendix C to the Exhibit A Statement of Work ("SOW") of the Master Services
Agreement entered into by and between (CHINESE CHARACTER) Microsoft (China) Co.,
Ltd. and (CHINESE CHARACTER) ATA Testing Authority, Beijing Co., Ltd. ("ATA") on
May 16, 2003. This Missing Exam Information and Data Feed Processes may be
updated by Microsoft from time to time and ATA shall obey the Missing Exam
Information in accordance with the Master Services Agreement and SOW.
The "candidate" provided in this Missing Exam Information and Data Feed
Processes shall have the same meaning as the "Candidate" as defined in the SOW.
APPENDIX C TO EXHIBIT A
APPENDIX C
MISSING EXAM INFORMATION AND DATA FEED PROCESSES
ABBREVIATIONS
APPENDIX C TO EXHIBIT A
OVERVIEW
Purpose of this document is to outline the procedures necessary to ensure that
MSFT has successfully received all exam records from the ESP and to eliminate
these issues from becoming candidate escalations. This issue typically becomes a
candidate escalation when the missing exam record prohibits a candidate from
receiving a welcome kit--resolution of candidate escalations is of the highest
priority. This document will also include procedures for: confirmation & error
files; bi-monthly reconciliations; weekly conference calls with THE CALL CENTER
support and MSFT Tier One; and issues with the MSFT server locking the ESP out.
This document will also cover the escalation process as it relates to these
issues.
POSSIBLE MISSING EXAM SCENARIOS
1. Candidate's record is not sent from TC to ESP.
2. Candidate's record is sent from ESP to MSFT but score is failed in MSFT and
ESP's DBs, yet the candidate has a copy of a passing score report.
3. Candidate's record is sent from TC to ESP, but is not sent to MSFT.
4. Candidate's record is sent from ESP to MSFT, but MS-Cert rejects the
record. Typically, when this happens ESP has an error message that details
why the records was not accepted at MSFT.
5. Candidate's record is sent from ESP to MSFT, but MS-Cert rejects the record
without issuing an error message.
In Scenarios #'s 1 through 4 it is the responsibility of the ESP to correct the
issue. In Scenario #5 it is the responsibility of the ESP to escalate to Tier
One for resolution (copying the appropriate VAM).
PROCESS WHEN MISSING EXAMS ARE ESCALATED TO THE CALL CENTER--THIS IS THE HIGHEST
PRIORITY!
- - Missing exams are escalated to the call center from Candidates.
- - THE CALL CENTER will also enter these issues on their escalation
spreadsheet for tracking and recording dates and all communication with ESP
regarding the issue.
- - Depending on the priority of issue, the escalation will come from either
"MCP Hot," which is the highest priority, or from the dedicated THE CALL
CENTER contact (typically, lower priority issues are communicated via logs
or spreadsheets between THE CALL CENTER and the ESP).
- - If the escalation comes from "MCP Hot"
- The ESP must respond within 4 business hours, via email or by phone or
both confirming:
- Status of the issue and an expected resolution time/date.
- ESP to update THE CALL CENTER every 24 hours with status until
resolution with details of actions being taken to resolve issue.
- - ESP is expected to respond WITHIN 48 HOURS, via email, confirming with THE
CALL CENTER that the issue is either:
- Resolved & Resent
- Inform THE CALL CENTER that more info is needed (for example, a copy
of the score report).
- Update THE CALL CENTER on status of record in the event that a
specific issue demands more than 48 hours to resolve. If the issue is
that the candidate is suspected of cheating, the ESP should instruct
THE CALL CENTER that it is the ESP's issue and the ESP should follow
the requirements outlined in the Cheater Policy document.
- IN ALL INSTANCES, THE ESP MUST CORRESPOND CLOSELY WITH THE CALL
CENTER, VIA EMAIL AND ON THE WEEKLY CONFERENCE CALLS, UNTIL
RESOLUTION.
- - Identified missing exams should be resent to MSFT and ESP should verify
that the record was received and processed by MSFT, via the confirmation
file (see steps regarding the Confirmation File below).
- - All unresolved issues should be addressed during a regular conference call
that includes THE CALL CENTER, ESP and Tier One. During this call each
issue should be discussed in detail so THE CALL CENTER can appropriately
update MSFT and the customer as necessary on the status. This meeting
should not be used to necessarily raise new issues--all issues must be
escalated as soon as they are discovered, but this meeting should be used
to talk through difficult issues and discuss root cause analysis.
- - If the candidate is not suspected of cheating, the exam should be sent
twice. On the second error, the ESP should escalate to CertHelp. If
CertHelp cannot correct, CertHelp should manually recreate the record.
APPENDIX C TO EXHIBIT A
- - All action should be noted, in the escalation spreadsheet, by THE CALL
CENTER in order that MSFT has visibility to how often this happens. Also,
the ESP should work to identify the reason why the record would not import
into MS-Cert and update THE CALL CENTER with that information in order to
determine trends.
CONFIRMATION FILES
Unless an error occurs between the TC and the ESP or the ESP did not send the
record to MSFT, all escalations should be avoided by the ESP's daily monitoring
of the confirmation file.
- - ESP should monitor the confirmation files DAILY to ensure that all records
sent to MSFT are registered as either confirmed or rejected.
- - Exam results confirmation files are placed on the ESPs' Extranet share in
the "TO" directory every day. These files confirm whether the records were
received or rejected. The confirmation file lists all records received by
line item and distinguishes those that were successfully imported from
those that have an errors associated.
- - For those records that were rejected, ESP should correct the record and
re-send. If the ESP receives a second error on the same record, ESP is to
send details of the record (Reg ID# and error type) to Tier One via the
CertHelp alias.
- - Tier One will log the issue and attempt to resolve. If Tier One is unable
to resolve, Tier One will rebuild the exam to ensure that candidate is
taken care of, but will continue to escalate the issue to determine root
cause.
- - If the ESP receives an error that the ESP is unfamiliar with, the ESP
should escalate to Tier One with specific details of the record and error
message in question for direction from Tier One on how to handle the error.
MSFT WILL PUBLISH A LIST OF ALL ERRORS THAT MS-CERT CURRENTLY VALIDATES.
PROCESS FOR BI-MONTHLY RECONCILIATIONS
This process should be viewed as a back up process to the daily confirmation
process. The bi-monthly reconciliations should NOT uncover any new issues to the
ESP--all issues should be detected during the daily confirmation process and
either corrected daily or escalated immediately to CertHelp. If any records are
uncovered during this process and not previously escalated to MSFT, it should be
considered a red flag to the ESP and the VAM and indicate that the ESP is not
monitoring the daily confirmation process or unveil a larger problem with the
confirmation file process.
- - On a bi-monthly basis, the ESP will provide MSFT Tier One with a report of
all registration ID #'s from the previous month. By going back one month
for each reconciliation, each two week period will actually be reconciled
twice to catch any latent issues.
- - Tier One will query this against MS-Cert and determine if any records are
missing from MS-Cert.
- - Tier One will forward all missing registration ID's to ESP. The ESP should
correct and resend within 24 hours, emailing Tier One with notification of
the resend (always copying the appropriate VAM).
- - ESP should also build a report using the missing registration ID #'s with
candidate information and date the exam was taken and current status of
each issue.
- - ESP is responsible to verify that the records imported properly the second
time--this should be accomplished via the confirmation files.
- - If the ESP discovers that the records were originally sent to MSFT
correctly and did not generate appropriate confirmation files, ESP should
email Tier One with specific examples of records that were sent to MSFT and
did not generate the appropriate confirmation/error files. Tier One should
research to determine root cause of issue.
WEEKLY CONFERENCE CALLS
On a weekly basis, ESP, Tier One, THE CALL CENTER and appropriate VAM (as
necessary) should meet via conference call to discuss all open issues, including
those on THE CALL CENTER's open issues list and those records identified by the
regular reconciliations. The VAM will coordinate setting up these meetings with
the ESP. From these meetings THE CALL CENTER will update status on all open
issues and escalate as necessary.
APPENDIX C TO EXHIBIT A
DATA FEED PROCESS
- - In the event that ESP was unable to send exam records to MSFT on time, an
auto-email notification will go out from MSFT to members of MSFT and ESP
notifying both parties that the MS-Cert did not receive records for the
previous night.
- - ESP will contact MSFT regarding reason why records were not sent. In the
event the loss of exams is the result of a larger issue, ESP will also
update MSFT at this time.
- - In the event the ESP is unable to post files because the MSFT server will
not allow access, the ESP should contact the Business Partners Support Desk
at 425.703. ESP should also email CertHelp and copy Timothy Ricord at MS
OBS Support and inform them that they have contacted the Support Desk,
supplying the Log number if possible
REBUILDING OF EXAM RECORDS
In the event that an exam will not download into MS-Cert or a customer has
escalated a missing exam to MSFT and the issue can not be resolved quickly, the
exam record should be rebuilt by Tier One. To rebuild an exam, Tier One needs
the following information
1. MCP ID:
2. Exam Number:
3. Exam Date:
4. Exam Score:
5. Vendor Id (1=Sylvan, 8=Vue):
6. Vendor Registration Id:
7. Vendor Loc Code:
8. Vendor Client Code (i.e. 070, 071, 075):
9. Country code
** The appropriate Account Manager should be cc'd on all communications unless
otherwise notified.
APPENDIX C TO EXHIBIT A
(CHINESE CHARACTER) (CHINESE CHARACTER)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
APPENDIX D TO EXHIBIT A -- - ATA EXAM RESULTS IMPORT FORMATS
This ATA Exam Results Import Formats is attached hereto as Appendix D to the
Exhibit A Statement of Work ("SOW") of the Master Services Agreement entered
into by and between (CHINESE CHARACTERS) Microsoft (China) Co., Ltd. and
(CHINESE CHARACTERS) ATA TESTING AUTHORITY, BEIJING CO., LTD. ("ATA") on May 16,
2003. This ATA Exam Results Import Formats will be updated by Microsoft from
time to time and ATA shall obey the ATA Exam Results Import Formats in
accordance with the Master Services Agreement and SOW.
The term"candidate" provided in this ATA Exam Results Import Formats shall have
the same meaning as the "Candidate" as defined in the SOW.
The terms "MCP EXAM PROVIDER", "MCP EXAM VENDOR", "EXAM VENDOR", and "VENDOR"
shall refer to ATA as defined in the SOW.
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
(CDDG IT LOGO)
ATA EXAM RESULTS IMPORT FORMATS
2
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
TABLE OF CONTENTS
3
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
1. EXECUTIVE SUMMARY
This Document contains the standardized data definition for data exchange
between Microsoft and the MCP exam providers. Currently the Certification
Data Staging (CDS) system is designed to receive and validate the data from
the exam vendors, making the data available to MSCert and ILR systems as
well as exporting confirmation and error/exception data back to the
vendors.
MSCert is a system that evaluates and certifies candidates whose
qualifications match the current MCP requirements.
ILR is a new "Item Level Response" system that requires similar data feeds
from the vendors.
1.1. WHAT IS CERTIFICATION DATA STAGING (CDS)?
CDS is a data warehouse intended to centralize the data feeds from the
MCP Exam vendors, making data available to downstream systems such as
MSCert and ILR.
1.2. CDS VENDOR INTERFACE
The CDS system acts as a data repository for downstream systems such
as MSCert and ILR.
1. CDS does data type validation
2. CDS sends confirmation records for good and error/exception
records to the vendors, e.g., 8 confirmation records out of 8
records received, good or bad.
3. CDS processes records from all vendors with the same format
containing:
- candidate profiles and profile changes
- exam results at both the exam and item levels which include
survey responses and comments
- merge/unmerge records exclusively for MSCert processing
- external certification records
- receipt confirmation records
- error/exception records
4. CDS gets receipt confirmation records from downstream systems
such as MSCert and ILR.
5. CDS gets error/exception records from downstream systems such as
MSCert and ILR.
6. CDS gets the corrected records from vendors.
4
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
1.3. INTERFACE GOALS
The goals of this interface are:
- To define a standardized data layout for continued data exchanged
between Microsoft and the vendors, and gain agreement between the
parties so that we have an efficient data interchange,
independent of what the exchange technology may be.
5
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
2. VENDOR FEEDS TO CDS
CDS would get daily feeds from vendors, in form of a zip file. More than
one zip file per vendor can be accepted by CDS, in a day.
We need to standardize the input content from all vendors so that we
receive multiple files of the same format and process only one batch of
records.
We expect the zip files to contain TAB delimited files, meeting the
following file naming convention as an example:
- CEmmddx.dat* for the CE data feed.
- SRmmddx.dat for the SR data feed.
* mmdd stands for month and day and x, the version number.
6
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
+ Used by ILR
OMEGA Used by MSCert
* Optional files
2.1. FILE HEADER FOR ALL FEEDS
For CDS importing requirements to be met each header record should contain
the 5 required fields plus the appropriate number of tabs such that the
total number of columns represented in the header record is equal to the
number of data columns in the file. For example, the CE record has 31
columns and the header record has 5. Therefore, tabs are required to fill
the remaining 26 unused columns on the header record. As for the SR record
that has 4 columns, no tabs will be required.
2.1.1. CE DATA - EXAM HEADER
7
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
2.1.2. SD DATA - EXAM SECTION DETAIL
2.1.3. IT DATA - EXAM ITEM DETAIL
8
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
2.1.4. SR DATA - SURVEY RESPONSE
Only get this data the first time someone takes the exam.
2.1.5. EC DATA - COMMENTS
2.1.6. CH DATA - PROFILE CHANGE
This is the Candidate Profile data for MSCert only.
9
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
10
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
2.1.7. XR DATA - EXTERNAL CERTIFICATION CHANGES
2.1.8. XC DATA - EXTERNAL CERTIFICATION CONFIRMATION/ERROR
2.1.9. PC DATA - PROFILE CONFIRMATION/ERROR
2.1.10. MR DATA - MERGE/UNMERGE
11
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
2.1.11. MC DATA - MERGE/UNMERGE CONFIRMATION/ERROR
12
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
3. CDS FEEDS TO VENDORS
This data contains receipt confirmation and data type validation error
records from CDS as well as rejected exception records from MSCert and
ILR.
We will be sending TAB delimited files, meeting the following file
naming convention as an example:
- MCEmmddx.dat for the MCE data.
- MPRmmddx.dat for the MPR data feed.
* mmdd stands for month and day and x, the version number.
3.1. RECORD HEADER FROM MICROSOFT
To be consistent with the inbound header requirement, each header
record will contain the 5 required fields plus the appropriate number
of tabs such that the total number of columns represented in the
header record is equal to the number of data columns in the file. For
example, the MCE record has 11 columns and the header
13
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
record has 5. Therefore, tabs are required to fill the remaining 6
unused columns on the header record. 13
* Vendor name here would be "Microsoft" or "MS"
3.1.1. MCE DATA - EXAM HEADER CONFIRMATION/ERROR FROM MICROSOFT
3.1.2. MPR DATA - PROFILE REQUEST FROM MICROSOFT
14
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
3.1.3. MXR DATA - EXTERNAL CERTIFICATION FROM MICROSOFT
15
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
3.1.4. MXC DATA - EXTERNAL CERTIFICATION CONFIRMATION/ERROR FROM
MICROSOFT
3.1.5. MPC DATA - PROFILE CONFIRMATION/ERROR FROM MICROSOFT
3.1.6. MMR DATA - MERGE/UNMERGE FROM MICROSOFT
3.1.7. MMC DATA - MERGE/UNMERGE CONFIRMATION/ERROR FROM MICROSOFT
16
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
4. ADDENDUM
4.1. SPECIAL ATTENTION
1. The ExamPhaseCode is paramount to the success of ILR. This field is
part of the definition of a candidate/exam and absolutely must be
reported accurately.
2. Prescore, Rescore and Live data should be returned in the daily feeds.
Currently, only Live data is returned daily and special requests are
made for Prescore and Rescore.
3. Currently, Prometric reports XML responses (from IIT exams) in a
separate file with an .MSX extension. XML data needs to be returned in
the proper file in the proper column.
4. Currently, Vue is stripping and escaping characters from the XML
responses. Nothing should be escaped and the only things that should
be stripped are line feeds, carriage returns and tabs (white space).
The non-white-space XML must be returned intact. Stripping out double
quotes (") and escaping tabs will invalidate the XML responses.
5. All items associated with a CandidateExam must be within the IT feed.
This will include 'display' items, non-viewed items, etc. Currently
some items are being stripped from the feed file based on criteria
that cannot be tracked within the feed files.
6. Exam results sent without the item-level detail associated will be
problematic, resulting as exceptions sent back to the vendor.
4.2. LIST OF ERROR NUMBERS/MESSAGES FROM CDS
Following is the categorization of error numbers generated by CDS:
17
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
Following is the list of error numbers that are generated by CDS and sent
to the vendors, and their meaning:
18
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
4.3. FILE LEVEL CONFIRMATION
Apart from the record level confirmation, CDS also creates and sends out
file level confirmation. This confirmation file contains the list of files
received and processed by CDS.
Sample of a confirmation file:
++
19
ATA EXAM RESULTS IMPORT FORMATS
++ = redacted item
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
20
Appendix E to Exhibit A: REPORT KEY
This Report Key is attached hereto as Appendix E to the Exhibit A Statement of
Work ("SOW") of the Master Services Agreement entered into by and between
(CHINESE CHARACTERS) Microsoft (China) Co., Ltd. and(CHINESE CHARACTERS) ATA
Testing Authority, Beijing Co., Ltd. ("ATA") on May 16, 2003. This Report Key
may be updated by Microsoft from time to time and ATA shall obey the Report Key
in accordance with the Master Services Agreement and SOW.
The term "candidate" provided in this Report Key shall have the same meaning as
the "Candidate" as defined in the SOW.
1
Appendix E to Exhibit A Report key
Report Key
2
Appendix E to Exhibit A Report key
3
Appendix E to Exhibit A Report key
4
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
5
Appendix F - ATA Price List
++ = redacted item
APPENDIX F - ATA PRICE LIST
A. OVERVIEW
This section of the document describes the pricing, invoicing and payment
details between ATA and MCCL.
B. ROYALTY PAYMENT
The unit royalty invoiced to ATA will be as follows. Pricing is subject to
change by MCCL at MCCL's sole discretion.
Batch One:
Appendix F - ATA Price List
++ = redacted item
Batch Two:
MICROSOFT OFFICE XP COURSES
Appendix F - ATA Price List
++ = redacted item
C. REPORTING AND INVOICING
At the end of each financial month (defined in Appendix H of the SOW) ATA will
provide MCCL with full reporting of all activities as listed in Appendix L, ATA
Revenue Reconciliation Reports. All such reporting will be provided in the form
of an Excel workbook. All reports shall be timely, complete and accurate. MCCL
reserves the right to audit as provided in the Master Service Agreement and
Exhibits attached thereto.
On receipt of this information MCCL will generate an invoice for the total
Licensing Fees due based on the number of printed Courseware Materials shipped
in that month. The invoice will be sent to ATA for payment. ATA shall be fully
responsible for and liable to MCCL for the payment in full of the due amount,
net 45 days upon receipt of the invoice. MCCL shall not be responsible for any
invoices incurred by ATA and/or ATA's partners.
D. RETURN ALLOWANCE
MCCL will allow a 2% allowance on returns for the term of the Master Service
Agreement. Any returns over 2% will not be accepted. Returns will be reported to
MCCL at month end as listed in Appendix L, ATA Revenue Reconciliation Report.
The returned units will be deducted from the total revenue units shipped to
calculate the net units shipped for the given month.
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
Appendix G to Exhibit A
CONTRACT NUMBER: ____________
MICROSOFT (CHINA) CO., LTD.
INDEPENDENT CONTRACTOR AGREEMENT
This Independent Contractor Agreement (the "Agreement") is attached hereto
as Appendix G to the Exhibit A Statement of Work ("SOW") of the Master Services
Agreement entered into by and between (CHINESE CHARACTERS) MICROSOFT (CHINA)
CO., LTD. ("MICROSOFT CHINA") and (CHINESE CHARACTERS)ATA TESTING AUTHORITY,
BEIJING CO., LTD. ("CONTRACTOR") on May 16, 2003. ATA shall obey this Agreement
in accordance with the Master Services Agreement and SOW.
MICROSOFT CHINA and CONTRACTOR agree as follows:
1. WORK
1.1 MICROSOFT CHINA hereby entrusts CONTRACTOR to carry out, complete and
deliver the work in accordance with the schedule set forth below in this
Agreement (the "WORK").
1.2 Being entrusted by MICROSOFT CHINA in accordance with the provisions of
this Agreement, CONTRACTOR shall carry out, complete and deliver the WORK in
accordance with the schedule set forth below and all WORK shall be completed on
or before June 30th 2004.
1.3 The WORK shall conform to the specifications and delivery schedule. In
the event that MICROSOFT CHINA desires to make changes to the WORK
specifications and/or delivery schedule during the term of this Agreement,
MICROSOFT CHINA shall so notify CONTRACTOR, and both parties shall agree in
writing on necessary adjustments, if any, to the other terms of this Agreement
required to accommodate such changes.
2. OWNERSHIP OF WORK BY MICROSOFT CHINA
To the extent that the WORK delivered to MICROSOFT CHINA hereunder includes
material subject to copyright, CONTRACTOR agrees that the WORK is done as a work
"entrusted" by MICROSOFT CHINA as the term "entrust" is used under the Chinese
copyright law, especially Article 17 of the Copyright Law of the People's
Republic of China (with the latest revisions by the National People's Congress
on October 27, 2001) and Article 11 of the Computer Software Protection
Regulation (promulgated by the State Council on December 20, 2001). CONTRACTOR
hereby agrees that MICROSOFT CHINA shall own, on an exclusive basis, all
copyrights in the WORK and all other rights in the intellectual property
developed by CONTRACTOR in the WORK. If MICROSOFT CHINA will not own all
copyrights and all other types of intellectual property rights in the WORK
(including patents, trade secrets, and other proprietary rights therein), for
whatever reasons, pursuant to the CONTRACTOR's such agreement as provided for in
the
1 of 8
Appendix G to Exhibit A
preceding sentence of this Section 2, to the extent that the WORK includes
material subject to copyright, patent, trade secret, or other proprietary right
protection, CONTRACTOR hereby assigns to MICROSOFT CHINA and its successors or
assignees all right, title and interest in and to the WORK, including all
copyrights, patents, trade secrets, and other proprietary rights therein
(including renewals thereof). CONTRACTOR shall execute and deliver such
instruments and take such other action as may be required and requested by
MICROSOFT CHINA to carry out the assignment contemplated by this paragraph. Any
documents, magnetically or optically encoded media, or other materials created
by CONTRACTOR pursuant to this Agreement shall be owned by MICROSOFT CHINA and
subject to the terms of this section.
3. CUSTOMER INFORMATION
3.1 CONTRACTOR acknowledges and agrees that customer information acquired
by CONTRACTOR, if any, in connection with the WORK, including, without
limitation, MICROSOFT CHINA customer lists and updates, personally identifiable
information (customer name, e-mail address, postal address, telephone number,
mother's maiden name, etc.), transactional data, sales and activity information,
and profile information (collectively, "Customer Information") shall be
considered proprietary information of MICROSOFT CHINA and all right, title and
interest in the Customer Information is owned by MICROSOFT CHINA.
3.2 CONTRACTOR shall use any such Customer Information only as necessary to
perform the WORK in accordance with this Agreement and shall maintain such
Customer Information in strict confidence in accordance with the provisions of
Section 6 hereinbelow. CONTRACTOR will not share any Customer Information that
is collected or possessed by CONTRACTOR with any third parties for any reason
whatsoever. If CONTRACTOR is served with a court order compelling disclosure of
any customer information, CONTRACTOR will oppose the order, will notify
MICROSOFT CHINA of such an order, and will provide MICROSOFT CHINA with the
opportunity to intervene before CONTRACTOR files any response to the order.
3.3 CONTRACTOR will take reasonable steps to protect Customer Information
in CONTRACTOR'S possession, if any, from unauthorized use, access, disclosure,
alteration or destruction. Security measures shall include access controls,
encryption or other means, where appropriate. CONTRACTOR agrees to conduct an
audit on at least an annual basis to evaluate the security of Customer
Information in CONTRACTOR'S possession, if any, and to verify that the terms of
this Agreement with respect to Customer Information are being followed.
3.4 CONTRACTOR shall abide by all applicable Chinese laws and regulations,
including the Anti-Unfair Competition Law, and international law and practice,
when CONTRACTOR is collecting, holding or processing any of the Customer
Information.
4. REPRESENTATIONS AND WARRANTIES
CONTRACTOR hereby represents and warrants that:
2 of 8
Appendix G to Exhibit A
(a) CONTRACTOR is a legal person duly established, validly existing and in
good standing under the laws of China;
(b) CONTRACTOR is properly holding a business license, the validity of
which will last throughout the term of this Agreement;
(c) CONTRACTOR possesses full legal right, power and authority, and has
taken all appropriate and necessary corporate action to authorize the execution
of this Agreement and to perform the obligations under this Agreement;
(d) CONTRACTOR is established with sufficient registered capital and will
have sufficient funds to carry out the obligations under this Contract;
(e) CONTRACTOR's performance of the WORK pursuant to this Agreement will
not violate any agreement or obligation between CONTRACTOR and a third party;
(f) the WORK will either be originally created by CONTRACTOR, or that
CONTRACTOR will obtain all necessary rights to the WORK to transfer ownership to
MICROSOFT CHINA as required by Section 2 hereinabove;
(g) the WORK will not infringe any copyright, patent, trade secret, or
other proprietary right held by any third party;
(h) all work provided by CONTRACTOR in connection with the WORK will be
performed in a professional manner and shall be of a high grade, nature, and
quality;
(i) CONTRACTOR warrants that CONTRACTOR has not and will not take any
actions that: (i) create, or purport to create, any obligation on behalf of
MICROSOFT CHINA, or (ii) grant, or purport to grant, any rights or immunities to
any third party under MICROSOFT CHINA's intellectual property or proprietary
rights. By way of example but without limitation to the foregoing, CONTRACTOR
shall not incorporate any Publicly Available Software in whole or in part into
any part of the WORK, or use Publicly Available Software in whole or in part in
the development of any part of the WORK in a manner that may subject the WORK,
in whole or in part, to all or part of the license obligations of any Publicly
Available Software. "Publicly Available Software" means any software that
requires as a condition of use, modification and/or distribution of such
software that such software or other software incorporated into, derived from or
distributed with such software (a) be disclosed or distributed in source code
form; (b) be licensed for the purpose of making derivative works; or (c) be
redistributable at no charge;
(j) CONTRACTOR is specifically advised that MICROSOFT CHINA is an equal
opportunity employer and promotes a diverse work environment. CONTRACTOR
understands that MICROSOFT CHINA will not tolerate any remark, gesture, or
behavior by CONTRACTOR that are discriminatory toward or offensive to
individuals on the basis of gender, race, color, national origin, age, religion,
or disability; and
3 of 8
Appendix G to Exhibit A
(k) if CONTRACTOR performs its obligations on MICROSOFT CHINA's property,
CONTRACTOR warrants that individuals placed at MICROSOFT CHINA have not been
convicted of a felony as an adult within the last seven (7) years.
5. INDEMNITY
CONTRACTOR agrees to indemnify, defend, and hold MICROSOFT CHINA and its
successors, officers, directors and employees harmless from any and all actions,
causes of action, claims, demands, costs, liabilities, expenses and damages
(including attorney's fees) arising out of, or in connection with (i) any claim
that the WORK infringes any copyright, patent, trade secret, trademark, or other
legal right of any third party, or (ii) any other claim that, if true, would
constitute a breach of CONTRACTOR's representations or warranties set forth in
Section 4 hereinabove.
6. CONFIDENTIALITY
6.1 CONTRACTOR agrees that at all times during the term of this Agreement,
and for five (5) years thereafter, CONTRACTOR will hold in strictest confidence,
and will not use or disclose to any third party, any confidential information of
MICROSOFT CHINA. The term "Confidential Information of MICROSOFT CHINA" shall
mean all non-public information that MICROSOFT CHINA designates as being
confidential, or which, under the circumstances of disclosure ought to be
treated as confidential. "Confidential Information of MICROSOFT CHINA" includes,
without limitation to, the terms and conditions of this Agreement, information
relating to software or hardware products released or unreleased by MICROSOFT
CHINA or the affiliates of MICROSOFT CHINA, marketing or promotion of any
product of MICROSOFT CHINA or of the affiliates of MICROSOFT CHINA, business
policies or practices of MICROSOFT CHINA or of the affiliates of MICROSOFT
CHINA, customers or suppliers of MICROSOFT CHINA or of the affiliates of
MICROSOFT CHINA, or information received from others that MICROSOFT CHINA or the
affiliates of MICROSOFT CHINA are obligated to treat as confidential. An
"affiliate" means any person, partnership, joint venture, corporation or other
form of enterprise, domestic or foreign, including but not limited to
subsidiary, that directly or indirectly, controls or is controlled by, or is
under common control of MICROSOFT CHINA. If CONTRACTOR has any questions as to
what comprises such confidential information, CONTRACTOR agrees to consult with
MICROSOFT CHINA. "Confidential Information of MICROSOFT CHINA" shall not include
information that was known to CONTRACTOR prior to MICROSOFT CHINA's disclosure
to CONTRACTOR, or information that becomes publicly available through no fault
of CONTRACTOR.
6.2 Press Releases/Publicity. CONTRACTOR shall not issue press releases or
publicity that relates to this Agreement. Such press statements shall be
released by MICROSOFT CHINA, at its sole discretion. CONTRACTOR shall not use
the name "Microsoft", "MICROSOFT CHINA", or "MICROSOFT CORPORATION" in any kinds
of brochures, advertisements, websites, news releases, publicity or
presentations to any third parties. CONTRACTOR may, however, with the prior
written consent of MICROSOFT CHINA, use the name "MICROSOFT CHINA" in client
presentations
4 of 8
Appendix G to Exhibit A
and/or in written response to requests for client lists as part of the Request
for Proposals, Requests for Information, etc.
6. TERMINATION
6.1 MICROSOFT CHINA may terminate this Agreement and cancel CONTRACTOR's
obligations hereunder at any time without cause and without further obligation
to CONTRACTOR except for payment due for obligations performed prior to the date
of such termination (not to exceed the amount payable for the WORK milestone
immediately following the last accepted WORK milestone, if any). In the event of
termination of this Agreement and upon request by MICROSOFT CHINA, CONTRACTOR
agrees to turn over to MICROSOFT CHINA all Confidential Information of MICROSOFT
CHINA and Customer Information in CONTRACTOR's possession, and all WORK, whether
completed or in progress, within ten (10) days of such termination.
6.2 In the event of termination of this Agreement for any reason, Sections
2, 3, 4, 5, 6, 8 and 9.6 of this Agreement shall survive termination.
8. INSURANCE
8.1 CONTRACTOR warrants that it shall maintain sufficient insurance
coverage to enable it to meet its obligations created by this Agreement and by
law, and the Contractor shall for the term of this Agreement be insured to the
full extent of any potential liability, loss or damages relating to the
performance of the Contractor's obligations pursuant to this Agreement. Without
limiting the foregoing, to the extent available in China, CONTRACTOR warrants
that such insurance shall include the following lines of coverage (with minimum
limits equal to One Million United States Dollars (US$1,000,000.00) per
occurrence) to the extent this Agreement creates exposures generally covered by
these insurance policies: Commercial General Liability (Occurrence Form),
Automobile Liability, Workers' Compensation (statutory limits), and Employer's
Liability.
8.2 In addition, if:
(a) the WORK consists of, includes, or results in any intellectual
property (including, without limitation to, material capable of being protected
by copyright, trade secret, or trademark law); or
(b) CONTRACTOR's performance of the WORK constitutes such professional
consulting work as:
(i) system development, design, or maintenance;
(ii) technical support; or
(iii) financial services (e.g., collections and transaction
authorization)
then CONTRACTOR shall maintain, to the extent available in China, Professional
Liability and Errors & Omissions Liability Insurance with policy limits of not
less than Two Million Dollars (US$2,000,000.00) for each claim with a deductible
of no more than Twenty-five Thousand Dollars (US$25,000.00). Such insurance
shall include coverage for
5 of 8
Appendix G to Exhibit A
infringement of any proprietary right of any third party, including but without
limitation to copyright, trade secret, and trademark infringement as related to
CONTRACTOR's performance under this Agreement. The Professional Liability and
Errors & Omissions Liability Insurance retroactive coverage date will be no
later than the Effective Date. CONTRACTOR shall maintain an extended reporting
period providing that claims first made and reported to the insurance by
CONTRACTOR within one (1) year after MICROSOFT CHINA's final payment for the
WORK will be deemed to have been made during the policy period.
8.3 Upon request, CONTRACTOR shall deliver to MICROSOFT CHINA proof of such
coverage. In the event that CONTRACTOR's proof evidences coverage which
MICROSOFT CHINA reasonably determines to be less than that required to meet
CONTRACTOR's obligations created by this Agreement, then CONTRACTOR agrees that
it shall promptly acquire such coverage and notify MICROSOFT CHINA in writing
thereof.
9. GENERAL
9.1 CONTRACTOR agrees that upon termination of this Agreement, CONTRACTOR
will return to MICROSOFT CHINA all drawings, blueprints, notes, memoranda,
specifications, software, electronic media, designs, devices, documents
including on-line documentation, and any other material containing or disclosing
any confidential or proprietary information of MICROSOFT CHINA. CONTRACTOR will
not retain any such materials without MICROSOFT CHINA's written approval.
9.2 CONTRACTOR is an independent contractor for MICROSOFT CHINA. Nothing in
this Agreement shall be construed as creating an employer-employee relationship
or principal-agent relationship, as a guarantee of future employment, or as a
limitation upon MICROSOFT CHINA's sole discretion to terminate this Agreement at
any time without cause. CONTRACTOR further agrees to be responsible for all of
CONTRACTOR's state and local taxes, withholding, social security, insurance, and
other benefits. Upon request, CONTRACTOR shall provide MICROSOFT CHINA with
satisfactory proof of independent contractor status (including a valid business
license). In the event that taxes are required to be withheld on payments made
hereunder, MICROSOFT CHINA may deduct such taxes from the amount owed to
CONTRACTOR and pay them to the appropriate taxing authority. MICROSOFT CHINA
shall in turn promptly secure and deliver to CONTRACTOR an official receipt for
any taxes withheld. MICROSOFT CHINA will use reasonable efforts to minimize such
taxes to the extent permissible under applicable law.
9.3 CONTRACTOR is free to engage in other independent contracting
activities, provided that CONTRACTOR does not engage in any such activities
which are inconsistent with any provisions hereof, or that so occupy
CONTRACTOR's time as to interfere with the proper and efficient performance of
CONTRACTOR's obligations hereunder.
9.4 CONTRACTOR may not assign this Agreement, or any rights or obligations
hereunder, whether by operation of contract, law, or otherwise, except with the
express
6 of 8
Appendix G to Exhibit A
written consent of MICROSOFT CHINA, and any attempted assignment by CONTRACTOR
in violation of this Section shall be void. For the purposes of this Agreement,
an "assignment" by CONTRACTOR under this Section shall be deemed to take place
in situations including, but without limitation to, each of the following: (a) a
change in beneficial ownership of CONTRACTOR, greater than that of twenty
percent (20%) (whether in a single transaction or series of transactions); (b) a
merger of CONTRACTOR with another party, whether or not CONTRACTOR is the
surviving entity; (c) the acquisition of more than twenty percent (20%) of
CONTRACTOR's equity ownership by another party (whether in a single transaction
or series of transactions); and (d) the sale or other transfer of more than
fifty percent (50%) of CONTRACTOR's assets (whether in a single transaction or
series of transactions). In the event of such assignment or attempted assignment
by CONTRACTOR, MICROSOFT CHINA shall have the right to immediately terminate
this Agreement.
9.5 CONTRACTOR agrees that MICROSOFT CHINA will not be responsible for loss
of or damage to any personal property located on MICROSOFT CHINA premises
belonging to CONTRACTOR.
9.6 The formation, validity, interpretation and performance of this
Contract and the disputes arising under this Contract shall be governed by the
published and publicly available laws of the People's Republic of China.
If a dispute arises in connection with the interpretation or implementation
of this Agreement, either party may submit the dispute for arbitration to the
China International Economic and Trade Arbitration Commission (the "CIETAC") for
arbitration in accordance with the rules of arbitration of CIETAC. The place of
arbitration will be Beijing, the People's Republic of China. The arbitration
proceedings shall be conducted in English. The arbitral award will be final and
binding on both parties.
There shall be three (3) arbitrators. Within thirty (30) days after giving
or receiving the demand for arbitration, CONTRACTOR and MICROSOFT CHINA shall
each select one (1) arbitrator. The third arbitrator shall be appointed by the
two arbitrators within thirty (30) days after both arbitrators have been
selected, or, failing agreement, the appointment shall be made as provided in
the Rules. Such third arbitrator shall be a non-Chinese and non-American citizen
who is able to use English as his/her working language.
If either party employs attorneys to enforce any rights arising out of or
relating to this Agreement, the prevailing party will be entitled to recover
reasonable costs and attorneys' fees.
9.7 This Agreement does not constitute an offer by MICROSOFT CHINA and it
shall not be effective until signed by both parties. This Agreement constitutes
the entire agreement between parties with respect to the subject matter hereof
and merges all prior and contemporaneous communications. It shall not be
modified except by a written agreement signed by CONTRACTOR and MICROSOFT CHINA
by their duly authorized representatives.
7 of 8
Appendix G to Exhibit A
9.8 This Agreement shall be executed in English and in Chinese versions
simultaneously in [___] counterparts in each language. In case of any
inconsistency between the two versions, the English version shall prevail.
IN WITNESS WHEREOF, each party hereto has caused this Agreement to be executed
by its duly authorized representative on the date first written above.
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA)CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD.
- ------------------------------------- ----------------------------------------
By (Sign) By (Sign)
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
- ------------------------------------- ----------------------------------------
[official seal] [official seal]
8 of 8
Appendix H to Exhibit A
MCCL - FISCAL CALENDAR FY'03
4-4-5 Week Calendar
Appendix H to Exhibit A
Appendix H to Exhibit A
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
Appendix I to Exhibit A
STANDARD PUBLISHING SCHEDULE
THE FOLLOWING DATES AND INFORMATION ARE FOR ILLUSTRATIVE PURPOSES ONLY AND
SUBJECT TO CHANGE.
Appendix I to Exhibit A
STANDARD PUBLISHING SCHEDULE
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
Appendix J to Exhibit A
MICROSOFT CORPORATION NON-DISCLOSURE AGREEMENT
(STANDARD RECIPROCAL)
This Non-Disclosure Agreement (the "Agreement") is made and entered into as
of the later of the two signature dates below by and between (CHINESE
CHARACTERS) Microsoft (China) Co., Ltd., LTD, a PRC corporation ("Microsoft"),
and (CHINESE CHARACTERS) ATA Testing Authority, Beijing Co., Ltd. ("ATA"), a PRC
corporation ("Company").
IN CONSIDERATION OF THE MUTUAL PROMISES AND COVENANTS CONTAINED IN THIS
AGREEMENT AND THE MUTUAL DISCLOSURE OF CONFIDENTIAL INFORMATION, THE PARTIES
HERETO AGREE AS FOLLOWS:
1. Definition of Confidential Information and Exclusions.
(a) "Confidential Information" means nonpublic information that a party to
this Agreement ("Disclosing Party") designates as being confidential to the
party that receives such information ("Receiving Party") or which, under the
circumstances surrounding disclosure ought to be treated as confidential by the
Receiving Party. "Confidential Information" includes, without limitation,
information in tangible or intangible form relating to and/or including released
or unreleased Disclosing Party software or hardware products, the marketing or
promotion of any Disclosing Party product, Disclosing Party's business policies
or practices, and information received from others that Disclosing Party is
obligated to treat as confidential. Except as otherwise indicated in this
Agreement, the term "Disclosing Party" also includes all Affiliates of the
Disclosing Party and, except as otherwise indicated, the term "Receiving Party"
also includes all Affiliates of the Receiving Party. An "Affiliate" means any
person, partnership, joint venture, corporation or other form of enterprise,
domestic or foreign, including but not limited to subsidiaries, that directly or
indirectly, control, are controlled by, or are under common control with a
party.
(b) Confidential Information shall not include any information, however
designated, that: (i) is or subsequently becomes publicly available without
Receiving Party's breach of any obligation owed Disclosing Party; (ii) became
known to Receiving Party prior to Disclosing Party's disclosure of such
information to Receiving Party pursuant to the terms of this Agreement; (iii)
became known to Receiving Party from a source other than Disclosing Party other
than by the breach of an obligation of confidentiality owed to Disclosing Party;
(iv) is independently developed by Receiving Party; or (v) constitutes Feedback
(as defined in Section 5 of this Agreement).
2. Obligations Regarding Confidential Information
(a) Receiving Party shall:
(i) Refrain from disclosing any Confidential Information of the
Disclosing Party to third parties for five (5) years following
the date that Disclosing Party first discloses such Confidential
Information to Receiving Party, except as expressly provided in
Sections 2(b) and 2(c) of this Agreement;
(ii) Take reasonable security precautions, at least as great as the
precautions it takes to protect its own confidential information,
but no less than reasonable care, to keep confidential the
Confidential Information of the Disclosing Party;
(iii) Refrain from disclosing, reproducing, summarizing and/or
distributing Confidential Information of the Disclosing Party
except in pursuance of Receiving Party's business relationship
with Disclosing Party, and only as otherwise provided hereunder;
and
(iv) Refrain from reverse engineering, decompiling or disassembling
any software code and/or pre-release hardware devices disclosed
by Disclosing Party to Receiving Party under the terms of this
Agreement, except as expressly permitted by applicable law.
Appendix J to Exhibit A
(b) Receiving Party may disclose Confidential Information of Disclosing
Party in accordance with a judicial or other governmental order, provided that
Receiving Party either (i) gives the undersigned Disclosing Party reasonable
notice prior to such disclosure to allow Disclosing Party a reasonable
opportunity to seek a protective order or equivalent, or (ii) obtains written
assurance from the applicable judicial or governmental entity that it will
afford the Confidential Information the highest level of protection afforded
under applicable law or regulation. Notwithstanding the foregoing, the Receiving
Party shall not disclose any computer source code that contains Confidential
Information of the Disclosing Party in accordance with a judicial or other
governmental order unless it complies with the requirement set forth in
sub-section (i) of this Section 2(b).
(c) The undersigned Receiving Party may disclose Confidential Information
only to Receiving Party's employees and consultants on a need-to-know basis. The
undersigned Receiving Party will have executed or shall execute appropriate
written agreements with its employees and consultants sufficient to enable
Receiving Party to enforce all the provisions of this Agreement.
(d) Receiving Party shall notify the undersigned Disclosing Party
immediately upon discovery of any unauthorized use or disclosure of Confidential
Information or any other breach of this Agreement by Receiving Party and its
employees and consultants, and will cooperate with Disclosing Party in every
reasonable way to help Disclosing Party regain possession of the Confidential
Information and prevent its further unauthorized use or disclosure.
(e)Receiving Party shall, at Disclosing Party's request, return all
originals, copies, reproductions and summaries of Confidential Information and
all other tangible materials and devices provided to the Receiving Party as
Confidential Information, or at Disclosing Party's option, certify destruction
of the same.
3. Remedies
The parties acknowledge that monetary damages may not be a sufficient
remedy for unauthorized disclosure of Confidential Information and that
Disclosing Party shall be entitled, without waiving any other rights or
remedies, to such injunctive or equitable relief as may be deemed proper by a
court of competent jurisdiction.
4. Miscellaneous
(a) All Confidential Information is and shall remain the property of
Disclosing Party. By disclosing Confidential Information to Receiving Party,
Disclosing Party does not grant any express or implied right to Receiving Party
to or under any patents, copyrights, trademarks, or trade secret information
except as otherwise provided herein. Disclosing Party reserves without prejudice
the ability to protect its rights under any such patents, copyrights,
trademarks, or trade secrets except as otherwise provided herein.
(b) In the event that the Disclosing Party provides any computer software
and/or hardware to the Receiving Party as Confidential Information under the
terms of this Agreement, such computer software and/or hardware may only be used
by the Receiving Party for evaluation and providing Feedback (as defined in
Section 5 of this Agreement) to the Disclosing Party. Unless otherwise agreed by
the Disclosing Party and the Receiving Party, all such computer software and/or
hardware is provided "AS IS" without warranty of any kind, and Receiving Party
agrees that neither Disclosing Party nor its suppliers shall be liable for any
damages whatsoever arising from or relating to Receiving Party's use of or
inability to use such software and/or hardware.
(c) The parties agree to comply with all applicable international and
national laws that apply to (i) any Confidential Information, or (ii) any
product (or any part thereof), process or service that is the direct product of
the Confidential Information, including the U.S. Export Administration
Regulations, as well as end-user, end-use and destination restrictions issued by
U.S. and other governments. For additional information on exporting Microsoft
products, see http://www.microsoft.com/exporting/.
(d) The terms of confidentiality under this Agreement shall not be
construed to limit either the Disclosing Party or the Receiving Party's right to
independently develop or acquire products without use of the other party's
Appendix J to Exhibit A
Confidential Information. Further, the Receiving Party shall be free to use for
any purpose the residuals resulting from access to or work with the Confidential
Information of the Disclosing Party, provided that the Receiving Party shall not
disclose the Confidential Information except as expressly permitted pursuant to
the terms of this Agreement. The term "residuals" means information in
intangible form, which is retained in memory by persons who have had access to
the Confidential Information, including ideas, concepts, know-how or techniques
contained therein. The Receiving Party shall not have any obligation to limit or
restrict the assignment of such persons or to pay royalties for any work
resulting from the use of residuals. However, this sub-paragraph shall not be
deemed to grant to the Receiving Party a license under the Disclosing Party's
copyrights or patents.
(e) This Agreement constitutes the entire agreement between the parties
with respect to the subject matter hereof. It shall not be modified except by a
written agreement dated subsequent to the date of this Agreement and signed by
both parties. None of the provisions of this Agreement shall be deemed to have
been waived by any act or acquiescence on the part of Disclosing Party, the
Receiving Party, their agents, or employees, but only by an instrument in
writing signed by an authorized employee of Disclosing Party and the Receiving
Party. No waiver of any provision of this Agreement shall constitute a waiver of
any other provision(s) or of the same provision on another occasion.
(f) If either Disclosing Party or the Receiving Party employs attorneys to
enforce any rights arising out of or relating to this Agreement, the prevailing
party shall be entitled to recover reasonable attorneys' fees and costs. This
Agreement shall be construed and controlled by the laws of the People's Republic
of China, and the parties further consent to submit any disputation arising from
and/or in connection with this Agreement to China International Economic & Trade
Arbitration Commission (CIETAC) for arbitration in Beijing in accordance with
the Commission's arbitration rules in effect at the time of applying for
arbitration. The arbitral award is final and binding upon both parties.
(g) This Agreement shall be binding upon and inure to the benefit of each
party's respective successors and lawful assigns; provided, however, that
neither party may assign this Agreement (whether by operation of law, sale of
securities or assets, merger or otherwise), in whole or in part, without the
prior written approval of the other party. Any attempted assignment in violation
of this Section shall be void.
(h) If any provision of this Agreement shall be held by a court of
competent jurisdiction to be illegal, invalid or unenforceable, the remaining
provisions shall remain in full force and effect.
(i) Either party may terminate this Agreement with or without cause upon
ninety (90) days prior written notice to the other party. All sections of this
Agreement relating to the rights and obligations of the parties concerning
Confidential Information disclosed during the term of the Agreement shall
survive any such termination.
5. Suggestions and Feedback
The Receiving Party may from time to time provide suggestions, comments or
other feedback ("Feedback") to the Disclosing Party with respect to Confidential
Information provided originally by the Disclosing Party. Both parties agree that
all Feedback is and shall be given entirely voluntarily. Feedback, even if
designated as confidential by the party offering the Feedback, shall not, absent
a separate written agreement, create any confidentiality obligation for the
receiver of the Feedback. Receiving Party will not give Feedback that is subject
to license terms that seek to require any Disclosing Party product, technology,
service or documentation incorporating or derived from such Feedback, or any
Disclosing Party intellectual property, to be licensed or otherwise shared with
any third party. Furthermore, except as otherwise provided herein or in a
separate subsequent written agreement between the parties, the receiver of the
Feedback shall be free to use, disclose, reproduce, license or otherwise
distribute, and exploit the Feedback provided to it as it sees fit, entirely
without obligation or restriction of any kind on account of intellectual
property rights or otherwise.
================================================================================
Appendix J to Exhibit A
IN WITNESS WHEREOF, the parties hereto have executed this Agreement.
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD.
By: By:
--------------------------------- ------------------------------------
Name: Name:
------------------------------- ----------------------------------
Title: Title:
------------------------------ ---------------------------------
Date: Date:
------------------------------- ----------------------------------
Appendix K to Exhibit A
GUIDELINES FOR USING THE MICROSOFT CERTIFIED PROFESSIONAL
EXAM PROVIDER LOGO (the "GUIDELINES")
(MICROSOFT(R) CERTIFIED PROFESSIONAL LOGO)
EXAM PROVIDER
These GUIDELINES, being an integral part of the Master Service Agreement, are
issued to ATA Testing Authority, Beijing Co., Ltd. (referred to as "You" in the
following), for its use of the Logo (see the logo hereinabove) in performing its
obligations under the Master Service Agreement and the Exhibits thereto.
- - You must have signed the Master Service Agreement and Exhibit A SOW before
using the Microsoft Certified Professional Exam Provider Logo as specified
hereinabove (the "Logo").
- - You may only use the Logo on your Web site, marketing material, and
letterhead to indicate that You cooperate with Microsoft (China) Co., Ltd.
("MCCL") for the Ministry of Education Polytechnic Program.
- - MCCL will provide You with electronic or camera ready artwork of the Logo.
You may not alter this artwork in any way.
- - The Logo may not be translated or otherwise localized into any other
language. Any localized versions of the Logo must be provided by MCCL.
- - You may not display the Logo in a manner that implies that the Logo is a
part of your company or service name.
- - Your trade name or company name must appear on any materials where the Logo
is used. The Logo cannot appear larger than or more prominent than your
name, trademark, logo or trade name.
- - The Logo may not be used in any manner that expresses or might imply MCCL's
affiliation, sponsorship, endorsement, certification, or approval [of the
Ministry of Education Polytechnic Program], other than as set forth by the
Master Service Agreement and the Exhibits thereto.
- - The Logo, or any elements thereof, may not be included in your trade or
business name, domain name, product or service name, logo, trade dress,
design, slogan, or other trademarks.
- - You may not combine the Logo with any other object, including, but not
limited to, other logos, icons, words, graphics, photos, slogans, numbers,
design features, symbols or Web site audio files.
- - The Logo may not be imitated or used as a design feature in any of your
materials or on your Web site.
- - The Logo shall be attributed to Microsoft Corporation in all materials
where it is used, with the following attribution clause: "Microsoft is a
registered trademark of Microsoft Corporation in the United States and
other countries."
- - You may not use the Logo in any way other than as specified in these
GUIDELINES.
- - MCCL shall be entitled to supervise the quality of the services rendered by
You in using the Logo. Such right shall be exercised by MCCL directly, or
by a third party authorized by MCCL. You shall
Appendix K to Exhibit A
observe the quality standards set out by MCCL with respect to the services
rendered by You in using the Logo, and shall guarantee the quality of such
services.
- - You acknowledge that "Microsoft" and the "Microsoft" trademark are renowned
in China. You shall not engage in any activities which may have any adverse
effect on the reputation and goodwill of MCCL, its affiliates or the
relevant trademarks.
- - For the purpose of submitting the duplicate copy of these GUIDELINES and
the Master Services Agreement to the trademark bureau for recordation under
the Trademark Law of China, You shall provide MCCL with all necessary
assistance and support from time to time.
- - Upon expiration of the Master Services Agreement, or in the event that the
Logo is discontinued and is not created, You will not be permitted to use
any of Microsoft's trademarks and/or the Logo.
SPACING
The Logo must stand alone. A minimum amount of space must be left between the
Logo and any other object such as type, other logos, photography, borders,
edges, and so on. The required border or empty space around the Logo must be x
wide, where x equals the height of the graphic, as represented by the height of
the box which contains the words "Microsoft Certified".
MINIMUM SIZE
You should take care of maintain the integrity of all elements of the Logo. For
example, the logotype and trademark notations must be readable; in no case
should the Logo appear at such a small size that these conditions are not met.
The minimum acceptable size for the Logos is 1 inch (2.54cm) in length.
COLOR
The Microsoft Certified Professional Exam Provider Logo can appear in two colors
or in black and white.
MISUSES
Your use of the Logo in violation of any of the rules specified in these
GUIDELINES shall be deemed as specified misuses of Microsoft's trademarks and
Logos as contained in the Master Services Agreement, which must be corrected in
accordance with the Master Services Agreement.
TRADEMARK REGISTRATION NUMBER
"Microsoft" has been duly registered in the People's Republic of China.
Registration No. of "Microsoft" is 621,767 for class 16.
Microsoft is a registered trademark of Microsoft Corporation in the United
States and other countries.
2
Appendix K to Exhibit A
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date
3
Appendix L of Exhibit A
ATA REVENUE RECONCILIATION REPORTS
A. OVERVIEW
This section of the document describes the reports required to be sent from
ATA to Microsoft. The reports will be used by MS Finance for booking and
reconciliation of financial transactions occurring at the end of each
fiscal month. Reporting will be provided for each month, dates to be
defined.
B. TRANSMISSION FORMAT
- MS require electronic communications via e-mail attachment, or other
method that is mutually agreed upon.
- Electronic transmissions need to be in Microsoft Excel Workbook
format.
C. TRANSMISSION FREQUENCY
ATA must submit the report files to MS no later than 8:00 A.M Beijing Time
on the 5th day following the end of each financial month (defined in
Appendix H of the SOW).
D. FILE REQUIREMENTS
1. WORKBOOK
1. "UNITS" is defined as inventory that is physically present in the
warehouse, in any condition or state (including damaged, locked, allocated,
or inventory on a warehouse pick ticket).
Appendix L of Exhibit A
2. "BACK ORDER QUANTITY" is defined as an order which will, for any of the
following reasons, not be fulfilled immediately: the items being ordered
are not currently in stock, items being held for future release date, order
held for credit authorization, or orders held while customer on credit
hold.
3. "QTY DAMAGED" is defined as inventory in unsellable condition.
4. "QTY MISPRINT" is defined as inventory that contains printing errors.
Misprinted inventory will be locked, so that it cannot be allocated to
orders.
5. "QTY ORDERED, NOT YET ALLOCATED" is defined as inventory that has been
ordered, but not allocated (not backordered).
6. "QTY ALLOC. TO ORDERS NOT YET PICKED" is defined as units of physical
inventory that have been assigned to a sales or stock order (present in the
warehouse, not damaged, locked, on a picked, or included in backorders).
WORKSHEET 2 - RECEIPT
DESCRIPTION The number of copies received per ISBN per
receipt date during the fiscal month just
completed. Required whenever inventory is
received by ATA during the month from sources
other than customers (printers, other Press
warehouses, etc.)
WORKSHEET 3 - UNITS SHIPPED
DESCRIPTION Transactions per customer, per transaction
type, per line for the fiscal month.
Transaction detail should balance with Header
records. Welcome Kits should be included in
this reporting, and defined as Code 2: Sample
Sales.
Rounding errors can occur, so quantity*price/unit is not always equal to total
amount in the header tab.
Microsoft uses one and two byte codes to report month end financial transactions
in the workbook.
Appendix L of Exhibit A
WORKSHEET 4 - WELCOME KITS
DESCRIPTION Number of welcome kits shipped during the
fiscal month
#2 Number of new registrations for classes received this month. Only new
registrations should be reported.
WORKSHEET 6 - INVENTORY FORECAST
DESCRIPTION Forecast of sales by part number for next 3
months & next 6 months.
1. "UNITS" is defined as inventory that is physically present in the
warehouse, in any condition or state (including damaged, locked, allocated,
or inventory on a warehouse pick ticket).
2. "BACK ORDER QUANTITY" is defined as an order which will, for any of the
following reasons, not be fulfilled immediately: the items being ordered
are not currently in stock, items being held for future release date, order
held for credit authorization, or orders held while customer on credit
hold.
3. "USAGE" is defined as the estimated sales for the specified period of time.
4. "TITLE STATUS" is defined as the lifecycle status of the product, which
will be either NYP, not yet printed, or IP, in print. Once a title is
released and made available to customers it changes from NYP to IP.
Appendix L of Exhibit A
(CHINESE CHARACTERS) (CHINESE CHARACTERS)
MICROSOFT (CHINA) CO., LTD. ATA TESTING AUTHORITY, BEIJING CO., LTD
- ------------------------------------- ----------------------------------------
Signature Signature
- ------------------------------------- ----------------------------------------
Name (Print) Name (Print)
- ------------------------------------- ----------------------------------------
Title Title
- ------------------------------------- ----------------------------------------
Date Date