text:
Patent US7464849 - Electro-mechanical surgical instrument with closure system and anvil ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA surgical instrument. Various embodiments relate to a surgical cutting and fastening instrument that has an end effector with an anvil that can pivot between open and closed positions relative to an elongate channel that can support a staple cartridge therein. The anvil may be opened and closed by a...http://www.google.com/patents/US7464849?utm_source=gb-gplus-sharePatent US7464849 - Electro-mechanical surgical instrument with closure system and anvil alignment componentsAdvanced Patent SearchPublication numberUS7464849 B2Publication typeGrantApplication numberUS 11/344,021Publication dateDec 16, 2008Filing dateJan 31, 2006Priority dateJan 31, 2006Fee statusPaidAlso published asCN101011276A, CN101011276B, US20070175961Publication number11344021, 344021, US 7464849 B2, US 7464849B2, US-B2-7464849, US7464849 B2, US7464849B2InventorsFrederick E. Shelton, IV, Stephen J. Balek, Eugene L. TimpermanOriginal AssigneeEthicon Endo-Surgery, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (99), Non-Patent Citations (2), Referenced by (23), Classifications (13), Legal Events (3) External Links: USPTO, USPTO Assignment, EspacenetElectro-mechanical surgical instrument with closure system and anvil alignment componentsUS 7464849 B2Abstract A surgical instrument. Various embodiments relate to a surgical cutting and fastening instrument that has an end effector with an anvil that can pivot between open and closed positions relative to an elongate channel that can support a staple cartridge therein. The anvil may be opened and closed by a drive system. Various embodiments employ a powered drive system for opening and closing the anvil and powering a knife and staple drive assembly used to cut tissue and deploy the staples in the cartridge. Other embodiments employ a mechanical system to open and close the anvil and employ a powered system to power the knife and staple drive assembly. In various embodiments, the drive assembly serves to axially and laterally align anvil relative to the elongate channel. The anvil is constructed to move in a substantially parallel path relative to the elongate channel during closing.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS2037727Dec 27, 1934Apr 21, 1936United Shoe Machinery CorpFasteningUS3269630Apr 30, 1964Aug 30, 1966Harry FleischerStapling instrumentUS3551987Sep 12, 1968Jan 5, 1971Wilkinson Jack EStapling clamp for gastrointestinal surgeryUS3643851Aug 25, 1969Feb 22, 1972United States Surgical CorpSkin staplerUS3662939Feb 26, 1970May 16, 1972United States Surgical CorpSurgical stapler for skin and fasciaUS3717294Dec 14, 1970Feb 20, 1973Surgical CorpCartridge and powering instrument for stapling skin and fasciaUS3734207Dec 27, 1971May 22, 1973Fishbein MBattery powered orthopedic cutting toolUS3819100Sep 29, 1972Jun 25, 1974United States Surgical CorpSurgical stapling instrumentUS3892228Oct 3, 1973Jul 1, 1975Olympus Optical CoApparatus for adjusting the flexing of the bending section of an endoscopeUS3894174Jul 3, 1974Jul 8, 1975Emhart CorpInsulated staple and method of making the sameUS3940844Apr 4, 1974Mar 2, 1976Pci Group, Inc.Method of installing an insulating sleeve on a stapleUS4213562May 1, 1978Jul 22, 1980Roger GarrettProgrammerUS4331277May 23, 1980May 25, 1982United States Surgical CorporationSelf-contained gas powered surgical staplerUS4349028Oct 3, 1980Sep 14, 1982United States Surgical CorporationSurgical stapling apparatus having self-contained pneumatic system for completing manually initiated motion sequenceUS4383634May 26, 1981May 17, 1983United States Surgical CorporationSurgical stapler apparatus with pivotally mounted actuator assembliesUS4402445Oct 9, 1981Sep 6, 1983United States Surgical CorporationSurgical fastener and means for applying sameUS4415112Oct 27, 1981Nov 15, 1983United States Surgical CorporationSurgical stapling assembly having resiliently mounted anvilUS4475679Aug 7, 1981Oct 9, 1984Fleury Jr George JMulti-staple cartridge for surgical staplersUS4500024Jun 9, 1983Feb 19, 1985Ethicon, Inc.Multiple clip applierUS4520817Mar 8, 1982Jun 4, 1985United States Surgical CorporationSurgical instrumentsUS4522327May 18, 1983Jun 11, 1985United States Surgical CorporationSurgical fastener applying apparatusUS4526174Nov 7, 1983Jul 2, 1985Minnesota Mining And Manufacturing CompanyStaple and cartridge for use in a tissue stapling device and a tissue closing methodUS4527724Jun 10, 1983Jul 9, 1985Senmed, Inc.Disposable linear surgical stapling instrumentUS4530453Oct 4, 1983Jul 23, 1985United States Surgical CorporationSurgical fastener applying apparatusUS4566620Oct 19, 1984Jan 28, 1986United States Surgical CorporationArticulated surgical fastener applying apparatusUS4573622Oct 19, 1984Mar 4, 1986United States Surgical CorporationSurgical fastener applying apparatus with variable fastener arraysUS4580712Oct 19, 1984Apr 8, 1986United States Surgical CorporationSurgical fastener applying apparatus with progressive application of fastenerUS4605001Oct 19, 1984Aug 12, 1986Senmed, Inc.Surgical stapling instrument with dual staple height mechanismUS4606343Aug 18, 1980Aug 19, 1986United States Surgical CorporationSelf-powered surgical fastening instrumentUS4608981Oct 19, 1984Sep 2, 1986Senmed, Inc.Surgical stapling instrument with staple height adjusting mechanismUS4610383Oct 14, 1983Sep 9, 1986Senmed, Inc.Disposable linear surgical staplerUS4629107Jun 5, 1984Dec 16, 1986Vsesojuzny Nauchno-Issledovatelsky I Ispytatelny Institute Meditsinskoi TekhnikiLigating instrumentUS4671445Aug 9, 1984Jun 9, 1987Baxter Travenol Laboratories, Inc.Flexible surgical stapler assemblyUS4709120Jun 6, 1986Nov 24, 1987Pearson Dean CUnderground utility equipment vaultUS4715520Oct 10, 1985Dec 29, 1987United States Surgical CorporationSurgical fastener applying apparatus with tissue edge controlUS4728020Aug 30, 1985Mar 1, 1988United States Surgical CorporationArticulated surgical fastener applying apparatusUS4728876Feb 19, 1986Mar 1, 1988Minnesota Mining And Manufacturing CompanyOrthopedic drive assemblyUS4729260Dec 8, 1986Mar 8, 1988Desoutter LimitedTwo speed gearboxUS4754909Jun 5, 1987Jul 5, 1988Barker John MFlexible staplerUS4767044Oct 4, 1985Aug 30, 1988United States Surgical CorporationSurgical fastener applying apparatusUS4790225Oct 21, 1986Dec 13, 1988Panduit Corp.Dispenser of discrete cable ties provided on a continuous ribbon of cable tiesUS4869414Feb 29, 1988Sep 26, 1989United States Surgical CorporationArticulated surgical fastener applying apparatusUS4869415Sep 26, 1988Sep 26, 1989Ethicon, Inc.Energy storage means for a surgical staplerUS4880015Jun 3, 1988Nov 14, 1989Nierman David MBiopsy forcepsUS4892244Nov 7, 1988Jan 9, 1990Ethicon, Inc.Surgical stapler cartridge lockout deviceUS4938408Jan 9, 1989Jul 3, 1990Ethicon, Inc.Surgical stapler safety and sequencing mechanismsUS4941623Dec 26, 1989Jul 17, 1990United States Surgical CorporationStapling process and device for use on the mesentery of the abdomenUS4944443Apr 22, 1988Jul 31, 1990Innovative Surgical Devices, Inc.Surgical suturing instrument and methodUS4955959May 26, 1989Sep 11, 1990United States Surgical CorporationLocking mechanism for a surgical fastening apparatusUS4978049May 26, 1989Dec 18, 1990United States Surgical CorporationThree staple drive memberUS5027834May 1, 1990Jul 2, 1991United States Surgical CorporationStapling process for use on the mesenteries of the abdomenUS5031814Sep 10, 1990Jul 16, 1991United States Surgical CorporationLocking mechanism for surgical fastening apparatusUS5040715May 26, 1989Aug 20, 1991United States Surgical CorporationApparatus and method for placing staples in laparoscopic or endoscopic proceduresUS5061269Aug 21, 1990Oct 29, 1991Joseph J. BerkeSurgical rongeur power grip structure and methodUS5065929Apr 1, 1991Nov 19, 1991Ethicon, Inc.Surgical stapler with locking meansUS5080556Sep 28, 1990Jan 14, 1992General Electric CompanyThermal seal for a gas turbine spacer discUS5084057May 30, 1990Jan 28, 1992United States Surgical CorporationApparatus and method for applying surgical clips in laparoscopic or endoscopic proceduresUS5100420Jul 18, 1989Mar 31, 1992United States Surgical CorporationApparatus and method for applying surgical clips in laparoscopic or endoscopic proceduresUS5104025Sep 28, 1990Apr 14, 1992Ethicon, Inc.Intraluminal anastomotic surgical stapler with detached anvilUS5106008Jul 11, 1991Apr 21, 1992United States Surgical CorporationLocking mechanism for a surgical fastening apparatusUS5111987May 14, 1990May 12, 1992Moeinzadeh Manssour HSemi-disposable surgical staplerUS5116349May 23, 1990May 26, 1992United States Surgical CorporationSurgical fastener apparatusUS5129570Nov 30, 1990Jul 14, 1992Ethicon, Inc.Surgical staplerUS5137198May 16, 1991Aug 11, 1992Ethicon, Inc.Fast closure device for linear surgical stapling instrumentUS5156315Apr 26, 1991Oct 20, 1992United States Surgical CorporationArcuate apparatus for applying two-part surgical fastenersUS5171247Oct 17, 1991Dec 15, 1992Ethicon, Inc.Endoscopic multiple ligating clip applier with rotating shaftUS5171249Apr 4, 1991Dec 15, 1992Ethicon, Inc.Endoscopic multiple ligating clip applierUS5200280Apr 14, 1992Apr 6, 1993Black & Decker Inc.Terminal cover for a battery packUS5207697Jun 27, 1991May 4, 1993Stryker CorporationBattery powered surgical handpieceUS5211649Dec 12, 1988May 18, 1993Vaso Products Australia Pty. LimitedVenous cuff applicator, cartridge and cuffUS5240163Oct 30, 1991Aug 31, 1993American Cyanamid CompanyLinear surgical stapling instrumentUS5246156Sep 12, 1991Sep 21, 1993Ethicon, Inc.Multiple fire endoscopic stapling mechanismUS5253793Feb 6, 1991Oct 19, 1993United States Surgical CorporationApparatus for applying two-part surgical fastenersUS5275323Mar 20, 1992Jan 4, 1994Ethicon, Inc.Surgical staplerUS5307976Jul 20, 1992May 3, 1994Ethicon, Inc.Linear stapling mechanism with cutting meansUS5309927Oct 22, 1992May 10, 1994Ethicon, Inc.Circular stapler tissue retention spring methodUS5318221Sep 10, 1992Jun 7, 1994United States Surgical CorporationApparatus and method for placing staples in laparoscopic or endoscopic proceduresUS5330502Oct 9, 1992Jul 19, 1994Ethicon, Inc.Rotational endoscopic mechanism with jointed drive mechanismUS5332142Jun 24, 1993Jul 26, 1994Ethicon, Inc.Linear stapling mechanism with cutting meansUS5339799Apr 17, 1992Aug 23, 1994Olympus Optical Co., Ltd.Medical system for reproducing a state of contact of the treatment section in the operation unitUS5350400Jun 30, 1992Sep 27, 1994American Cyanamid CompanyMalleable, bioabsorbable, plastic staple; and method and apparatus for deforming such stapleUS5379933Jun 15, 1993Jan 10, 1995United States Surgical CorporationArcuate apparatus for applying two-part surgical fastenersUS5383880Dec 16, 1992Jan 24, 1995Ethicon, Inc.Endoscopic surgical system with sensing meansUS5383881Sep 22, 1993Jan 24, 1995United States Surgical CorporationSafety device for use with endoscopic instrumentationUS5389098May 14, 1993Feb 14, 1995Olympus Optical Co., Ltd.Surgical device for stapling and/or fastening body tissuesUS5391180Apr 21, 1993Feb 21, 1995United States Surgical CorporationArticulating endoscopic surgical apparatusUS5395033May 24, 1994Mar 7, 1995Ethicon, Inc.Endoscopic surgical instrument with electromagnetic sensorUS5395312May 10, 1993Mar 7, 1995Desai; AshvinSurgical toolUS5397046Mar 22, 1993Mar 14, 1995United States Surgical CorporationLockout mechanism for surgical apparatusUS5403312Jul 22, 1993Apr 4, 1995Ethicon, Inc.Electrosurgical hemostatic deviceUS5405344Sep 30, 1993Apr 11, 1995Ethicon, Inc.Articulable socket joint assembly for an endoscopic instrument for surgical fastner track thereforUS5409498Nov 5, 1992Apr 25, 1995Ethicon, Inc.Rotatable articulating endoscopic fastening instrumentUS5413267May 14, 1991May 9, 1995United States Surgical CorporationSurgical stapler with spent cartridge sensing and lockout meansUS5413268Sep 30, 1993May 9, 1995United States Surgical CorporationApparatus and method for placing stables in laparoscopic or endoscopic proceduresUS5415335Apr 7, 1994May 16, 1995Ethicon Endo-SurgerySurgical stapler cartridge containing lockout mechanismUS5425745Oct 29, 1993Jun 20, 1995United States Surgical CorporationApparatus and method for placing staples in laparoscopic or endoscopic proceduresUS5431322Nov 2, 1993Jul 11, 1995United States Surgical CorporationSelf contained gas powered surgical apparatusUS5431668Apr 29, 1993Jul 11, 1995Ethicon, Inc.Ligating clip applierUSRE34519Dec 31, 1991Jan 25, 1994Ethicon, Inc.Surgical stapler cartridge lockout deviceNon-Patent CitationsReference1Disclosed Anonymously, "Motor-Driven Surgical Stapler Improvements," Research Disclosure Database No. 526041, Published: Feb. 2008.2European Search Report, Application No. 07250393.1, dated Aug. 9, 2007 (8 pages).Referenced byCiting PatentFiling datePublication dateApplicantTitleUS7918377Oct 16, 2008Apr 5, 2011Ethicon Endo-Surgery, Inc.Surgical stapling instrument with apparatus for providing anvil position feedbackUS7950560 *Apr 13, 2007May 31, 2011Tyco Healthcare Group LpPowered surgical instrumentUS7954685 *Oct 30, 2008Jun 7, 2011Tyco Healthcare Group LpArticulation and firing force mechanismsUS7963433 *Sep 22, 2008Jun 21, 2011Tyco Healthcare Group LpSurgical device having multiple driversUS7988028Sep 3, 2009Aug 2, 2011Tyco Healthcare Group LpSurgical instrument having an asymmetric dynamic clamping memberUS8074859Mar 31, 2010Dec 13, 2011Tyco Healthcare Group LpSurgical instrumentUS8196795 *Aug 13, 2010Jun 12, 2012Ethicon Endo-Surgery, Inc.Disposable motor-driven loading unit for use with a surgical cutting and stapling apparatusUS8220688Dec 24, 2009Jul 17, 2012Ethicon Endo-Surgery, Inc.Motor-driven surgical cutting instrument with electric actuator directional control assemblyUS8235272 *Nov 20, 2009Aug 7, 2012Tyco Healthcare Group LpSurgical stapling device with captive anvilUS8245899Jan 26, 2010Aug 21, 2012Ethicon Endo-Surgery, Inc.Driven surgical stapler improvementsUS8267302Jul 7, 2011Sep 18, 2012Tyco Healthcare Group LpSurgical instrument and loading unit for use therewithUS8292150 *Aug 24, 2011Oct 23, 2012Tyco Healthcare Group LpAdapter for powered surgical devicesUS8360298Apr 14, 2010Jan 29, 2013Covidien LpSurgical instrument and loading unit for use therewithUS8444037Jul 9, 2012May 21, 2013Covidien LpSurgical stapling device with captive anvilUS8444038Oct 7, 2010May 21, 2013Covidien LpTissue stop for surgical instrumentUS8459525 *Feb 14, 2008Jun 11, 2013Ethicon Endo-Sugery, Inc.Motorized surgical cutting and fastening instrument having a magnetic drive train torque limiting deviceUS8540131Mar 15, 2011Sep 24, 2013Ethicon Endo-Surgery, Inc.Surgical staple cartridges with tissue tethers for manipulating divided tissue and methods of using sameUS20100301096 *Aug 13, 2010Dec 2, 2010Ethicon Endo-Surgery, Inc.Disposable motor-driven loading unit for use with a surgical cutting and stapling apparatusUS20110121050 *Nov 20, 2009May 26, 2011Nicholas David ASurgical stapling device with captive anvilUS20120104071 *Aug 24, 2011May 3, 2012Tyco Healthcare Group LpAdapter for powered surgical devicesWO2011078960A1Dec 6, 2010Jun 30, 2011Ethicon Endo-Surgery, Inc.Motor-driven surgical cutting instrument with electric actuator directional control assemblyWO2011084969A1Jan 5, 2011Jul 14, 2011Ethicon Endo-Surgery, Inc.Test device for a surgical toolWO2013122739A2Jan 30, 2013Aug 22, 2013Ethicon-Endo Surgery, Inc.Surgical cutting and fastening instrument with apparatus for determining cartridge and firing motion status* Cited by examinerClassifications U.S. Classification227/178.1, 227/179.1, 227/19, 227/175.1, 227/180.1International ClassificationA61B17/39Cooperative ClassificationA61B2017/00398, A61B17/07207, A61B2017/00685, A61B2017/0038, A61B2017/0688, A61B2017/07285European ClassificationA61B17/072BLegal EventsDateCodeEventDescriptionMay 16, 2012FPAYFee paymentYear of fee payment: 4Mar 6, 2012CCCertificate of correctionMay 1, 2006ASAssignmentOwner name: ETHICON ENDO-SURGERY, INC., OHIOFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHELTON, IV, FREDERICK E.;BALEK, STEPHEN J.;TIMPERMAN, EUGENE L.;REEL/FRAME:017552/0349;SIGNING DATES FROM 20060413 TO 20060421RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google


text:
Patent US7643995 - Method of automatically ranking speech dialog states and transitions to aid ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA method of identifying problems in a speech recognition application is provided and includes the step of obtaining a speech application call log containing log data on question-answer (QA) states and transitions. Then, in accordance with the method, for each of a multiple transitions between states,...http://www.google.com/patents/US7643995?utm_source=gb-gplus-sharePatent US7643995 - Method of automatically ranking speech dialog states and transitions to aid in performance analysis in speech applicationsAdvanced Patent SearchPublication numberUS7643995 B2Publication typeGrantApplication numberUS 11/054,096Publication dateJan 5, 2010Filing dateFeb 9, 2005Priority dateFeb 9, 2005Fee statusPaidAlso published asUS20060178883Publication number054096, 11054096, US 7643995 B2, US 7643995B2, US-B2-7643995, US7643995 B2, US7643995B2InventorsAlejandro Acero, Dong YuOriginal AssigneeMicrosoft CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (14), Non-Patent Citations (19), Referenced by (2), Classifications (9), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetMethod of automatically ranking speech dialog states and transitions to aid in performance analysis in speech applicationsUS 7643995 B2Abstract A method of identifying problems in a speech recognition application is provided and includes the step of obtaining a speech application call log containing log data on question-answer (QA) states and transitions. Then, in accordance with the method, for each of a multiple transitions between states, a parameter is generated which is indicative of a gain in a success rate of the speech recognition application if all calls passing through the transition passed instead through other transitions. In exemplary embodiments, the parameter is an Arc Cut Gain in Success Rate (ACGSR) parameter. Methods of generating the ACGSR, as well as systems and tools for aiding developers are also disclosed.
1. A method of identifying problems in a speech recognition application, the method comprising:
obtaining from a computer storage medium a speech application call log containing log data on question-answer (QA) states and transitions;
generating from the speech application call log, using a processor coupled to the computer storage medium, for each of a plurality of transitions between states in the speech application call log, a parameter which is indicative of a gain in a success rate of the speech recognition application if all calls passing through the transition passed instead through others of the plurality of transitions, wherein generating the parameter for each of the plurality of transitions comprises generating the parameter, for each of the plurality of transitions between the states, indicative of the gain in the success rate of the speech recognition application if all calls passing through the transition were instead evenly distributed between others of the plurality of transitions;
ranking, using the processor, the plurality of transitions as a function of their corresponding generated parameters, indicative of a gain in the success rate of the speech recognition application if all calls passing through the transition passed instead through others of the plurality of transitions, to identify transitions which will most improve performance of the speech recognition application; and
displaying on a display device the plurality of transitions ordered as a function of rank for use by a user in identifying problems in the speech recognition application in order to improve performance of the speech recognition application.
2. The method of claim 1, wherein generating the parameter for each of the plurality of transitions comprises:
generating the parameter, for each of the plurality of transitions between the states, such that it is indicative of the gain in the success rate of the speech recognition application if all calls passing through the transition passed instead through others of the plurality of transitions at a same success rate as other calls originally passing through the others of the plurality of transitions.
3. The method of claim 1, wherein the transitions correspond to arcs between states in a call transition diagram, and wherein the parameter generated for each of the plurality of transitions is an Arc Cut Gain in Success Rate (ACGSR).
determining for each of the plurality of arcs or transitions which calls passing through the arc or transition are sunken calls; and
wherein generating the parameter for each of the plurality of arcs or transitions further comprises generating the ACGSR for each of the arcs or transitions using only sunken calls for that arc or transition.
5. The method of claim 4, wherein determining for each of the plurality of arcs or transitions which calls passing through the arc or transition are sunken calls comprises:
determining which calls passing through the arc or transition are sunken calls based upon a timestamp.
6. The method of claim 4, wherein generating the ACGSR for each of the arcs or transitions comprises generating the ACGSR for each of the arcs or transitions in accordance with a relationship representable by the equation:
ACGSR  ( a ij ) = [ s i - s ij t i - t ij � t ij - s ij ] / N = ( s i � t ij - s ij � t ij - t i � s ij + t ij � s ij ) / [ N  ( t i - t ij ) ] = ( s i � t ij - t i � s ij ) / [ N  ( t i - t ij ) ] . where, aij is the arc or transition, ti is the total number of calls passing through a state qi, si as the total number of successful calls passing through the state qi, Tij is used to represent a set of calls associated with the arc aij, tij=|Tij| is used to represent the number of calls in the set, sij=|Sij| is used to represent the number of successful calls associated with the arc, and N is a total number of calls placed.
7. The method of claim 4, wherein generating the ACGSR for each of the arcs or transitions comprises generating the ACGSR for each of the arcs or transitions in accordance with a relationship representable by the equation:
ACGSR  ( a ij ) = [ s i - s ij t i - t ij � t ij - s ij ] / N = ( s i � t ij - s ij � t ij - t i � s ij + t ij � s ij ) / [ N  ( t i - t ij ) ] = ( s i � t ij - t i � s ij ) / [ N  ( t i - t ij ) ] . where, aij is the arc or transition, ti, is the total number of calls passing through a state qi, si as the total number of successful calls passing through the state qi, Tij is the set of sunken calls of the arc aij, tij =|T ij| is used to represent the number of calls in the set, sij=|Tij| is used to represent the number of successful sunken calls associated with the arc, and N is a total number of calls placed.
8. A computer storage medium having stored thereon computer-executable instructions for implementing speech recognition application problem identifying steps comprising:
accessing a speech application call log containing log data on question-answer (QA) states and transitions, wherein the transitions correspond to arcs between states in a call transition diagram;
generating, for each of a plurality of transitions between states, a parameter which is indicative of a gain in a success rate of the speech recognition application if all calls passing through the transition were routed instead through others of the plurality of transitions using a processor of the computer, wherein the parameter generated for each of the plurality of transitions is an Arc Cut Gain in Success Rate (ACGSR), and wherein generating the parameter for each of the plurality of arcs or transitions further comprises generating the ACGSR for each of the arcs or transitions using only sunken calls for that arc or transition.
9. The computer storage medium of claim 8, and having further stored thereon computer-executable instructions for implementing further speech recognition application problem identifying steps comprising:
ranking the plurality of transitions as a function of their corresponding generated parameters; and
displaying the plurality of transitions as a function of rank.
10. The computer storage medium of claim 9, wherein generating the parameter for each of the plurality of transitions comprises:
11. The computer storage medium of claim 9, wherein generating the parameter for each of the plurality of transitions comprises:
generating the parameter, for each of the plurality of transitions between the states, indicative of the gain in the success rate of the speech recognition application if all calls passing through the transition were instead evenly distributed between others of the plurality of transitions.
12. The computer storage medium of claim 8, wherein determining for each of the plurality of arcs or transitions which calls passing through the arc or transition are sunken calls comprises:
determining which calls passing through the arc or transition are sunken calls based upon a number associated an event.
13. The computer storage medium of claim 8, wherein generating the ACGSR for each of the arcs or transitions comprises generating the ACGSR for each of the arcs or transitions in accordance with a relationship representable by the equation:
14. The computer storage medium of claim 8, wherein generating the ACGSR for each of the arcs or transitions comprises generating the ACGSR for each of the arcs or transitions in accordance with a relationship representable by the equation:
ACGSR  ( a ij ) = [ s i - s ij t i - t ij � t ij - s ij ] / N = ( s i � t ij - s ij � t ij - t i � s ij + t ij � s ij ) / [ N  ( t i - t ij ) ] = ( s i � t ij - t i � s ij ) / [ N  ( t i - t ij ) ] . where, aij is the arc or transition, ti is the total number of calls passing through a state qi, si as the total number of successful calls passing through the state qi, Tij is the set of sunken calls of the arc aij, tij=|Tij| is used to represent the number of calls in the set, sij=|Sij| is used to represent the number of successful sunken calls associated with the arc, and N is a total number of calls placed.
CROSS-REFERENCE TO RELATED APPLICATIONS Reference is hereby made to the following co-pending and commonly assigned patent application: U.S. application Ser. No. 11/054,301, filed Feb. 9, 2005, entitled �INTERACTIVE CLUSTERING METHOD FOR IDENTIFYING PROBLEMS IN SPEECH APPLICATIONS�, which is hereby incorporated by reference in its entirety.
BACKGROUND OF THE INVENTION The present invention generally pertains to speech recognition applications and systems. More specifically, the present invention pertains to methods of identifying and analyzing performance problems during development of speech recognition applications.
Although great progress has been made in speech recognition technology during the past two decades, adoption of speech technology has not gone as smoothly as might be desired. This is evidenced by the relatively limited deployment of speech applications. The main barrier to wider adoption of speech recognition technology stems from the cost incurred in developing the speech applications. A typical speech application includes application logic, dialog, one or more grammars, a speech recognition engine, etc. Since it is unlikely that the system will be available right at the very beginning (i.e., without significant development), constructing a quality speech application usually involves four steps, namely, (1) design; (2) prototyping; (3) evaluation and tuning; and (4) deployment. Typically, the prototyping step and the evaluation and tuning step are performed iteratively, fine tuning the speech application through each iteration to improve performance.
Among these four steps, evaluation and tuning during test and pilot stages is one of the most important phases and usually costs most. The tuning phase can take many months and requires a team of developers, testers, and speech technology experts.
Significant efforts have been directed toward the goal of reducing the total cost needed to develop and deploy speech applications. For example, a series of development tools have recently been released by companies working in speech recognition. The development tools significantly improved the development process. However, some critical functionality is not available in these tools. For instance, speech application authors usually don't know what to do when the success rate of their application is not satisfactory, even though they have access to large quantities of application logs. It would therefore be very valuable to automatically (or semi-automatically) determine what callers are struggling with, or which parts of their application need the most work.
SUMMARY OF THE INVENTION A method of identifying problems in a speech recognition application is provided and includes the step of obtaining a speech application call log containing log data on question-answer (QA) states and transitions. Then, in accordance with the method, for each of a multiple transitions between states, a parameter is generated which is indicative of a gain in a success rate of the speech recognition application if all calls passing through the transition passed instead through other transitions. In exemplary embodiments, the parameter is an Arc Cut Gain in Success Rate (ACGSR) parameter. Methods of generating the ACGSR, as well as systems and tools for aiding developers are also disclosed.
FIG. 2 is a block diagram illustrating an algorithm for constructing a call transition diagram (CTD) from a call log.
FIG. 3 is a diagrammatic illustration of a tree type call-state graph (CSG).
FIG. 4 is a diagrammatic illustration of a CSG with loops and merges.
FIG. 5-1 is a block diagram illustrating a system or tool in accordance with an embodiment of the present invention.
FIG. 5-2 is a flow diagram illustrating a method in accordance with the present invention.
FIG. 6-1 is a diagrammatic illustration of an example of an initial interactive clustering (IC) step in which all calls are clustered into one unlabelled class.
FIG. 6-2 is a diagrammatic illustration of an example of a later interactive clustering (IC) step in which, after six calls are labeled, calls are automatically clustered into labeled classes and an unlabeled class.
FIG. 7 is a diagrammatic illustration of a call flow graph.
FIG. 8 is a screen shot of a user interface (UI) of a system or tool of the present invention.
FIG. 9 includes two tables illustrating performance evaluation data.
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS 1. Exemplary Environments
Various aspects of the present invention pertain to methods and apparatus for identifying problems in speech recognition applications. The methods and apparatus are useful in reducing the speech recognition application development time and to enhance performance of the applications. Both of these results can lead to reduced development costs and wider use of speech recognition technology.
Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not in any way limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, and set top boxes. Embodiments of the present invention can be implemented in the development of a wide variety of speech recognition applications, for example including voice-dialing systems, call routing systems, voice messaging systems, order management systems, or any application using dialog turns (i.e., question-answer or �QA� states and transitions) and a speech recognition engine to recognize speech from a user to guide the user to a desired result. These are simply examples of systems with which the present invention can be used to improve the development process.
It should be noted that the present invention can be carried out on a computer system such as that described with respect to FIG. 1. However, the present invention can be alternatively carried out on a server, a computer devoted to message handling, a distributed system in which different portions of the present invention are carried out on different parts of the distributed computing system, etc.
2. Speech Application Problem Identification Methods and System
The present invention includes methods, apparatus and systems which can be used to reduce the time and efforts needed by developers to identify problems in their applications during the evaluation and tuning phase of speech application development. It is assumed that developers have access to the speech application logs containing data on Question Answer (QA) states and transitions. The present invention can then be used to aid developers by mining and analyzing the log data. While in embodiments of the present invention the logs are referred to as �call logs�, inferring for example that the speech recognition application can be part of a telephone based system (for example a voice dialing or call center application), the invention is not limited to this type of application or to use with this type of logs. Use of the phrase �call logs� is intended to represent logs pertaining to telephone calls interacting with the speech recognition application, as well as any logs of interaction (�calls�) between a user and a speech recognition application, whether telephone based or not.
The approach used in some embodiments of the invention includes two primary steps. In the first step, calls are clustered based on QA states and transitions, and important QA states and transitions are identified based on a new criterion referred to here as Arc Cut Gain in Success Rate (ACGSR). The invention also includes an algorithm or method to estimate the ACGSR by analyzing the Call Transition Diagram (CTD) automatically inferred from the speech application log data. In the second step, calls associated with important QA transitions are clustered through a new approach referred to here as Interactive Clustering (IC). The purpose of IC is to automatically cluster calls that are similar to those labeled by the users, so that users can focus on checking those unlabeled calls and identifying new problems. Experiments on an auto-attendant application named �MS Connect� show that the approaches of the present invention can dramatically reduce the time and efforts needed to identify problems in speech applications.
The above concepts and methods are described below as follows: First, the step or method of clustering calls based on QA states and transitions, and ranking QA transitions, is described. Specifically, the concept of ACGSR for measuring the importance of the QA transitions is introduced, and an algorithm to estimate ACGSR is provided. Then, the step or method of identifying problems in the calls associated with those important QA transitions is illustrated. The concept of IC is described, and semi-supervised mixture model based algorithms for it are derived/disclosed. Finally, the performance evaluation of the algorithms on MS Connect log is reported. It will be shown that the present invention does significantly reduce the time and efforts needed by developers to identify problems in their applications.
II. Cluster and Rank QA Transitions
In this section, the step or method of clustering calls based on QA states and transitions, and ranking QA transitions based on ACGSR, is described. First, the concept of a call transition diagram (CTD) is introduced, as is a way to automatically construct a CTD from speech application logs. The concept of ACGSR is then described, and an ACGSR estimation algorithm is derived/disclosed. After that, issues related to presenting the ranking results are discussed. It is emphasized that the ultimate goal of performance evaluation and tuning is to improve the success rate of a speech application.
Definition 1: The success rate r is defined as the percentage of successful calls, i.e., calls that fulfilled their tasks, over all calls placed: r=S/N.
Here S is the number of calls which fulfilled their tasks and N is the total number of calls placed. Fulfillment of a task is defined by the corresponding speech application(s), and for example can be calls transferred in an auto-attendant application, or balance checked in an automatic support application.
A. Call Transition Diagram and its Construction
Definition 2: A Call Transition Diagram (CTD) is a 3-tuple CTD=(Q, A, T), where:
Q is a finite set of states Q={qi|i=1, . . . , I}; A is a finite set of arcs that A={aij}⊂(Q�Q); T is the set of calls passing through the arcs: T:A→SC, where C is the set of all calls placed C={cn|n=1, . . . , N}and SC is the superset of the set C.
Note that the CTD differentiates itself from the traditional Call Flow Diagram (CFD) in that the CTD contains the set of the calls passing through each arc. The CTD is also different from the traditional Markov Model description of the Call states. In the Traditional Markov Model description, the number of times a state and/or arc is visited determines the model parameters, while in the CTD the number of distinguished calls visiting each arc is measured. No matter how many times the same call has visited the same arc, it's counted as one call in the CTD. The reason for this difference is that more interest is placed in the successful calls rather than the events.
To make the following discussion easier, Tij is used to represent the set of calls associated with the arc aij, and tij=|Tij| is used to represent the number of calls in the set. Similarly, Sij is used to represent the set of successful calls associated with the arc aij, and sij=|Sij| is used to represent the number of successful calls associated with the arc.
The construction of the CTD from the application logs is straight forward. Table I shown in FIG. 2 summarizes an algorithm for performing such a construction. Note that, during the CTD construction process, calls are clustered based on the QA states and transitions they have visited.
B. ACGSR and Its Estimation
Once the CTD is obtained, those QA states and transitions that need the most attention from developers must be identified. One method of identifying these QA states is to list the QA states and transitions based on the local unsuccessful rate. This approach, however, is not optimal since the improvement in the local success rate does not necessary translate into the improvement in the overall success rate. For example, a QA state at the later stage with a high unsuccessful rate may not be worth working on since the total gain achieved by improving this QA state might be very limited. This suggests that a good criterion is critical to the effectiveness of the ranking. As mentioned previously, the criterion proposed here is referred to as ACGSR.
Definition 3: The Arc Cut Gain in Success Rate (ACGSR) is the change of success rate if all calls originally passing through an arc are distributed to other arcs with the same start state (as if the arc is cut from the CTD) given that all other system parameters are unchanged.
ACGSR essentially measures how important it is to reduce the number of calls passing through a specific arc. The higher the ACGSR, the more important it is. Unfortunately, ACGSR is not directly available from the CTD (or logs) and needs to be estimated. In accordance with one embodiment, to estimate ACGSR, it is assumed that calls redirected to other arcs have the same success rate as the calls originally passing through those arcs. If the CTD is a tree (with no loop and no merge) as represented in the call-state graph (CSG) 300 shown in FIG. 3, ACGSR of arc aij (designated by reference number 302) between states qi and qj (designated by reference numbers 304 and 306, respectively) can be estimated using Equation 1:
ACGSR  ( a ij ) = [ ∑ k ≠ j  s ik ∑ k ≠ j  t ik � t ij - s ij ] / N Equation   1 Defining ti as the total number of calls passing through the state qi, and si as the total number of success calls passing through the state qi, i.e.,
t i = ∑ k  t ik Equation   2 s i = ∑ k  s ik Equation   3 Equation 1 thus becomes Equation 4:
ACGSR  ( a ij ) = [ s i - s ij t i - t ij � t ij - s ij ] / N = ( s i � t ij - s ij � t ij - t i � s ij + t ij � s ij ) / [ N  ( t i - t ij ) ] = ( s i � t ij - t i � s ij ) / [ N  ( t i - t ij ) ] . Equation   4 To summarize, for the tree type CTD represented by CSG 300 illustrated in FIG. 3, estimation of the ACGSR of arc aij is based upon the scenario where calls originally passing through of arc aij are now redirected to other arcs (such as the arcs designated by reference numbers 308, 310, 312 and 314) whose start state is qi.
When the CTD is not a tree, however, Equation 4 does not hold any more due to the fact that the same call may flow out of a state several times through different arcs. Consider as an example the CTD represented by CSG 325 shown in FIG. 4 having incoming calls 326 entering state qi (represented by reference number 333) and passing through arcs aik, aij and aii (designated by reference numbers 327, 329 and/or 331) in route to states qk and qj (designated by reference numbers 335 and 337). This CSG includes loops and merges. Assume that calls 1, 2, and 3 passing through arc aij are failed and calls 4 and 5 passing through arc aik are successful. It can be seen that the success rate does not change if the arc aii is cut since all calls passing though aii have already been counted in the arcs aij and aik. However, if the arc aij is cut, all calls originally failed are now turned into successful calls and a big gain in the success rate is achieved.
The example illustrated in FIG. 4 suggests that only the calls flowing out of the start state 333 (no looping back) through the arc should be considered when estimating the ACGSR.
Definition 4: A call loops back to the start state qi of the arc aij from the end state qj if there is a list of arcs {ax k y k |k=1, . . . , K} such that x0=i, xk+1=yk, yK=j, and the call passes through them sequentially.
Definition 5: A sunken call of the arc aij is a call that eventually passes through arc aij and does not loop back to the start state qi again.
Theorem 1: A call c passing through the non-stop state qi is a sunken call of one and only one arc whose start state is qi.
Proof: Since the state is a non-stop state, the call must go out of the state qi and ends at another state. This means that c is a sunken call of at least one arc whose start state is qi. For the same reason, the call can't be a sunken call of multiple arcs with the same start state.
Definition 6: A timestamp is a number associated with each event. Later events have higher timestamps than the earlier ones. A timestamp can be real time or any number (e.g., event ID) that increases over time. In an example embodiment, time(c, aij) is used to indicate the timestamps of the event that the call c passes through the arc aij. Since each call may pass through the same arc several times, max(time(c, aij)) is used to indicate the latest time c passes through aij.
Theorem 2: A call c associated with arij is a sunken call of aij if and only if max(time(c, aij))>max(time(c, aki))∀k
Proof: If call c is a sunken call, then the call eventually flows out through arc aij and does not loop back again. This means that later (higher timestamp) events that are associated with call c do not occur in all arcs whose end state is qi.
Contradictory proof: If call c is not a sunken call, then there is a list of arcs {ax k y k |k=1, . . . , K} such that x0=i, xk+1=yk, yK=j, and the call passes through them sequentially. Referring to Equation 5, this means:
max  ( time  ( c , a ij ) ) <  time  ( c , a jy 1 ) <  time  ( c , a y 1  y 2 ) <  � <  time  ( c , a y K  i ) ≤  max  ( time  ( c , a y K  i ) ) .  Equation   5 The final algorithm of ACGSR estimation can be obtained by redefining Tij (recall from above that tij=|Tij|) as the set of sunken calls of the arc aij in Equation 5. Then, Equation 4 still holds for calculation of ACGSR.
C. Present Ranking Results to the Users
To present the result to developers, arcs are sorted on the estimated ACGSR in descending order. The higher the ACGSR for an arc or transition, the larger the gain in success rate which can be achieved by improving the QA state and transition. In addition, two other issues should be considered:
First, developers are interested in knowing how they may improve the system. For this reason, those arcs with negative ACGSR are not informative to them and should not be displayed.
Second, developers cannot manage to reduce the number of calls passing through some arcs even though those arcs may have large ACGSR values. For example, an application may have a menu for the callers to select from. Transitions to any state associated with an item in the menu should not be cut. Two solutions to this problem are provided: Developers can either mark these arcs in the CSG before running the tool or method of the present invention, or select to hide the transition in the displayed report.
The ACGSR estimation algorithm is not only useful in determining the most important QA states and/or transitions for developers to work on, but is also useful in many other areas. For example, by changing the grammar used in a QA state, the call flow would diverge from the original path. The ACGSR estimation algorithm can be used to estimate the total success rate gain after the grammar change without running the updated application.
D. General System and Method
FIG. 5-1 is a block diagram illustrating general system 500 configured to implement the methods of the present invention described above. System 500 includes an ACGSR generating component 504 configured to implement one or more of the above-described methods to generate the ACGSR for transitions or arcs using data in call log 502 pertaining to question-answer (QA) states and transitions. In some embodiments, component 504 can also generate call transition diagrams using, for example, the algorithm shown in FIG. 2. Also, in other embodiments, component 504 generates parameters other than the ACGSR as described above. However, in exemplary embodiments, at least one of these other parameters would still be indicative of a gain in the success rate of the speech recognition application if all calls passing through a transition passed instead through others of a plurality of transitions.
In system 500 illustrated in FIG. 5-1, a ranking component receives the ACGSR scores or other parameters, and ranks the transitions based on these scores or parameters. Then, graphical user interface (GUI) generating component 508 generates a GUI to display the transitions to a user as a function of rank. An example of such a GUI is described below with reference to FIG. 8.
FIG. 5-2 is a flow diagram 520 illustrating in more general terms a method as described above. As can be seen at block 522, the method includes obtaining a speech application call log containing log data on QA states and transitions. At block 524, the method is shown to include the step of generating a parameter (e.g., the ACGSR) for each desired transition which is indicative of the gain in success rate of the application if all calls passing through the transition passed instead through other transitions. Then, the method optionally includes the steps shown at blocks 526 and 528 of ranking or sorting the transitions as a function of their generated parameters, and displaying the transitions as a function of their rank. These steps are optional in some embodiments because it is possible that the generated ACGSRs or other parameters could be used in an automated fashion to correct identified problems without displaying the ranked transitions to a user.
III. Identify Problems through Interactive Clustering
Once the most important QA states and/or transitions are identified, it is desirable to group calls passing through these arcs into clusters based on the cause of problems. Ideally, the whole process should be automatic. However, there are several difficulties of doing so. First, it is impossible to know all possible causes of the problems, and it is therefore impossible to train classifiers for them before hand. Second, different applications may log different information and it is the additional information that separates one type of problem from another. Third, the parameters in different applications are different and so it's difficult to generate a set of universal classifiers.
For these reasons, the present invention addresses the problem from a different perspective, aiming to reduce the time and efforts needed for the developers to identify problems through Interactive Clustering (IC).
Definition 7: Interactive Clustering (IC) is a clustering process based on semi-supervised clustering technology. During initialization, an IC method or component clusters calls based on prior knowledge such as pre-trained classifiers. If no prior knowledge is available, IC keeps all calls in one cluster or groups them with unsupervised clustering. The developer checks unlabeled calls and labels them (with the cause of the problem, for example). The developer's interaction is used as supervision and/or feedback for IC to adjust and label other calls. This process goes on until all calls are labeled and the developer is satisfied with the labels.
The benefit of IC is clear. With a brute force approach, the developer needs to check, for example, 400 calls associated with a special QA transition to identify most of the problems. During this process, the developer frequently will become frustrated since he/she is checking calls with the same causes again and again. A slightly better approach is random sampling in which it is hard to determine the number of samples needed. A large number of samples typically requires lots of redundant work, and a small number of samples typically results in a low probability of finding all problems.
With IC, things are different. Once the developer labels a call, all similar calls are grouped together and labeled. The developer only needs to focus on those unlabeled calls which are usually associated with new causes. The number of calls the developer needs to check overall is thus greatly reduced.
FIGS. 6-1 and 6-2 illustrate an example of the IC method. Originally, all calls 350 (represented as circles) are unlabeled and clustered into one unlabeled class 352 as indicated in FIG. 6-1. After six calls are labeled (represented as darkened circles 350-1 through 350-6), calls are clustered into three classes, two of which are labeled (designated by reference numbers 354 and 356) as indicated in FIG. 6-2. The next call the developer needs to check is from the unlabeled class 352. The main component of IC is a semi-supervised, finite mixture distribution model based, clustering algorithm.
A. Membership Determination
The goal is to find the best classification for each call in which instances similar to labeled data are labeled accordingly the same or similarly, given a large set of unlabeled data instances and a small set of labeled data instances. For the sake of easy discussion, a variable is denoted by a capitalized token (e.g., Xi), and the value of the variable by that same token in lower case (e.g., xi). We use p(x|y) to denote the probability that X=x given Y=y or a probability distribution for X given Y.
The disclosed approach to cluster the whole data set X={xi|i=1 . . . , I} into J clusters C={cj|j=1, . . . , J} is mixture distribution model based. Given the model parameters λ, the probability that the instance xi belongs to class cj is:
p  ( c j ❘ x i , λ ) = p  ( c j ❘ λ )  p  ( x i ❘ c j , λ ) ∑ j ′ = 1 J  p  ( c j ′ ❘ λ )  p  ( x i ❘ c j ′ , λ ) , Equation   6 where p(xi|cj, λ) is the probability that the instance xi is generated by the class cj and p(cj|λ) is the prior of different classes. It is assumed that the prior follows Bernoulli distribution, i.e., without knowing the value of the data instance, the probability that it belongs to class cj is:
p(c j|λ)=πj� Equation 7
p  ( c j ❘ x i , λ ) = π j  p  ( x i ❘ c j , λ ) ∑ j ′ = 1 J  π j ′  p  ( x i ❘ c j ′ , λ ) Equation   8 The probabilities p(cj|xi, λ) are called membership probability. Once these probabilities have been computed, the data instance can either be assigned to the cluster with the highest probability (a hard assignment), or the data instance can be assigned fractionally to the set of clusters according to this distribution (a soft assignment). In exemplary embodiments, when the result is presented to the user, hard assignment is used.
B. Model Parameter Learning
The model parameters are estimated with an Expectation-Maximization (EM) algorithm. The EM algorithm starts with a set of initial values of the parameters, and then iterates between an Expectation (or E) step and a Maximization (or M) step until the parameter values converge to stable values. Given the model parameters λ, the probability that the instance xi is generated by the model is:
p  ( x i ❘ λ ) = ∑ j = 1 J  p  ( c j ❘ λ )  p  ( x i ❘ c j , λ ) = ∑ j = 1 J  π j  p  ( x i ❘ c j , λ ) . Equation   9 The probability that all data instances are generated by the model is:
p  ( X ❘ λ ) = ∏ i = 1 I  p  ( x i ❘ λ ) = ∏ i = 1 I  ∑ j = 1 J  π j  p  ( x i ❘ c j , λ ) . Equation   10 The posterior probability of model parameter λ is:
p  ( λ ❘ X ) = p  ( X ❘ λ )  p  ( λ ) p  ( X ) , Equation   11 where p(λ) is the prior distribution of λ. The maximum a posteriori (MAP) parameter is defined as:
λ MAP = arg   max λ  p  ( λ ❘ X ) = arg   max λ  p  ( X ❘ λ )  p  ( λ ) = arg   max λ  ( log   p  ( X ❘ λ ) + log   p  ( λ ) ) . Equation   12 When used in conjunction with vague or non-informative priors, MAP estimates are smoothed versions of Maximum Likelihood (ML) estimates. In one embodiment, Dirichlet distribution is chosen as the prior for the multinomial distribution with parameters φ=(φl, . . . , φN)
p  ( ϕ 1 , ... , ϕ N ❘ α 1 , ... , α N ) = Γ  ( ∑ n = 1 N  α n ) ∏ n = 1 N  Γ  ( α n )  ∏ n = 1 N  ϕ n α n - 1 Equation   13 In one embodiment, αl= . . . =αN=l/N is chosen.
Given the current parameters λ, the new parameters λ′ is thus estimated to maximize the following function:
Q  ( λ ′ , λ ) = E  ( p  ( λ ′ ❘ X ) ) = ∑ i = 1 I  ∑ j = 1 J  p  ( c j ❘ x i , λ )  [ log   p  ( X ❘ λ ′ ) + log   p  ( λ ′ ) ] . Equation   14 Assume that each data instance is an ordered vector of K attribute values xi={xik|k=1, . . . K}. It is further assumed that attributes are independent with each other, and therefore:
p  ( x i ❘ c j , λ ) = ∏ k = 1 K  p  ( x ik ❘ c j , λ ) Equation   15 Each attribute can have either nominal values or real values. Multinomial value attributes are modeled with Bernoulli distribution: λik={qjkl, . . . , qjkl k }. Real value attributes are modeled with Gaussian distribution: λjk={μjk, σjk}.
Maximizing the Q function with respect to each subset of parameters, the algorithms to update parameters can be obtained.
For prior of class distribution:
π ′ j = ∑ i = 1 I  p  ( c j ❘ x i , λ ) + 1 / J ∑ j ′ = 1 J  ∑ i = 1 I  p  ( c j ′ ❘ x i , λ ) + 1 Equation   16 For multinomial (Bernoulli distribution) parameters:
q ′ jkl = ∑ i = 1 I  p  ( c j ❘ x i , λ )  δ  ( x ik , l ) + 1 / L k ∑ j ′ = 1 J  ∑ i = 1 I  p  ( c j ′ ❘ x i , λ )  δ  ( x ik , l ) + 1   where : Equation   17 δ  ( x , v ) = { 1 , if   x = v 0 , otherwise Equation   18 For Gaussian parameters:
μ jk ′ = ∑ i = 1 I  p  ( c j ❘ x i , λ )  x ik ∑ j ′ = 1 J  ∑ i = 1 I  p  ( c j ′ ❘ x i , λ )  x ik Equation   19 σ jk ′ = ∑ i = 1 I  p  ( c j ❘ x i , λ )  ( x ik - μ jk ′ ) ∑ j ′ = 1 J  ∑ i = 1 I  p  ( c j ′ ❘ x i , λ ) + 1 Equation   20 At the initialization step, random probabilities are assigned to p(cj|xi, λ) which is used to estimate the initial model.
C. Supervision As Constraint and Feedback
Thus far, the clustering algorithm has been discussed without any labeled data. To use the data instances labeled by users, in some embodiments, the importance of the data instance is artificially boosted to the estimation of the model parameters:
p(c j |x i, λ)=B�δ(j, Lab(i)) Equation 21
where Lab(i) is the label of data point xi, and B is the boost factor. Boost factor B is originally set to 1 and doubles its value every time the converged result does not have labeled data correctly classified. It is straight forward to see that the labeled data is used both as constraint and as feedback in this approach.
Since it is a goal to help users to focus on calls that are not similar to those already labeled, the number of clusters in the clustering algorithm is set to one plus the number of known classes. In other words, J is set to 1 initially and increases by 1 each time a new class is discovered by the user. The IC method or algorithm stops when the number of instances in the unlabeled cluster is zero and the user is satisfied with the labels.
IV. Experiment and Evaluation
The present invention also includes a tool or component, which implements the above-described approach, for use by developers. This tool has been evaluated on the MS Connect application. MS Connect is an auto-attendant application based on the Microsoft Speech Server platform. It allows users to speak the name of the person they wish to contact and provides them with a variety of messaging options (transfer to office, transfer to cell, or leave a voice recording as an email attachment). When used to evaluate the tool, the MS Connect Service had 33 QA states and 137 transitions, and serves to transfer calls to more than fifty thousand employees of Microsoft Corporation. FIG. 7 illustrates part of the Call Flow Graph 400.
For performance monitoring and tuning purposes, all calls are logged until they are either transferred or hanged up. In the experiments used to evaluate the tool of the present invention, log data collected in June 2004 was used.
FIG. 8 is a screenshot or snapshot illustrating a graphical user interface (GUI) 550 of the tool. In this particular configuration of GUI 550, pane 560 is included, showing QA transitions sorted based on ACGSR scores (from highest ACGSR at the top to lowest ACGSR at the bottom). Each transition shown is followed by two sets of numbers. The first set of numbers indicates the absolute and percentage gain of ACGSR. The second set of numbers indicates the absolute and percentage of calls passing through the arc. The user can select a transition and QA in pane 560 using an input (such as a mouse click) from an input device. In the illustrated example, transition 562 is selected as represented by its highlighting in the display. In pane 570, calls associated with selected transition (in this case transition 562) are displayed. Again, the user can select a call in pane 570. In the illustrated example, call 572 (having Call ID 18830 is selected). In pane 580, the Call QA Summary for a selected call 572 is illustrated.
From FIG. 8, it can be seen that the transition which needs most work (having the highest ACGSR is the transition 564 from Phase1AskQA to FarEnd (hanged up by the caller). From pane 560, it can be seen that more than 10% (or 222) of all calls pass through this transition. As can also be seen in pane 560, if the calls flowing through this transition can be eliminated completely, an absolute 8.5% (or 189 calls) improvement in the success rate can be achieved. This information clearly indicates to the developer that this transition should be worked on, as it will have a significant impact on the performance of the system.
Another interesting finding is that 1.5% of all calls pass through the transition 566 from SayCallingContact to NearEnd (hanged up by the system) which was unexpected. Fixing problems in this transition results in a 1.5% gain in success rate. As can be seen from this experimental data, ACGSR clearly identified those QA states and transitions that developers should pay most attention to.
Evaluating the effectiveness of IC methods requires additional work. Traditionally, clustering algorithms are evaluated with criteria such as confusion matrix, precision, recall, F1 measure, balancing, purity, entropy, and mutual information. These criteria do not fit here since the purpose of IC is to identify all problems as fast as possible. Two obvious criteria are:
Criterion 1: The PPI is the Percentage of Problems Identified over all problems existing under the assumption that the developer can label each call he/she has checked. The best PPI possible is 100%.
Criterion 2: The ANCL (Average Number of Calls Labeled) is the total number of calls labeled manually over all problems identified. The best possible ANCL is 1.
Another criterion is:
Criterion 3: The Problem Distribution Accuracy (PDA) is the accuracy of the number of calls in each class. It is defined as the K-L (Kullback-Leibler) distance between the problem distribution obtained from IC and the true problem distribution:
PDA = ∑ i  p ^ i  log  ( p ^ i / p i ) , Equation   22 where {circumflex over (p)}i is the estimated percentage of calls in class i, and pi is the true percentage of calls in the class i. The best possible PDA is 0. See S. Kullback & R. Leibler, �On Information and Sufficiency�, Annals of Mathematical Statistics 22, 79-86, 1951.
PDA is related to the final goal of the analysis since it tells developers the order of problems they should work on and the estimated success rate gain after resolving each problem.
We evaluated IC using two sets of data generated from the ACGSR ranking and clustering step. The first set of data contains 459 calls classified into four types of problems with distributions 338, 36, 53, and 32. The second set of data contains 896 calls in four clusters each of which has 706, 69, 67, and 54 calls. The feature set we used in this experiment includes promptID, prompt barged in status, DtmfID, repetition of passing, duration, and engine confidence scores. We ran four independent experiments for each data set and compared IC with the brute force and random sampling approach. Tables 2 and 3 shown in FIG. 9 illustrate the results.
The brute force approach guarantees the correct identification of all problems and distributions. However, it requires checking of 115 calls in data set one and 224 calls in data set two for each problem identified. The random sampling approach gives better results. To compare apples to apples with IC, two sampling configurations were conducted. The first approach is to fix the sampling size (equivalent to ANL fix) and the second approach is to fix the percentage of problems identified (PPI) on average. The result shows that with the same ANL, IC can detect 93.8% of problems in both data sets and the sampling approach can only detect 62.5% of problems. To detect 93.8% of problems, IC requires checking of only 1.63 calls per problem in both data sets, while the sampling approach requires checking 14 calls per problem in data set one and 21 calls per problem in data set two. Furthermore, ANL increases very slowly using IC and much faster using the sampling approach with larger data sets. In both configurations, IC provides more accurate distribution than the sampling approach.
In conclusion, the methods of the present invention have been described in the context of log analysis and its application to speech application problem identification. It has been shown that using ACGSR as the criterion to rank the QA state and transitions, and using IC to help developers find problems quicker, can significantly shorten and improve the development process. Specifically, it has been demonstrated that the present invention can significantly reduce the time and effort required of developers to identify problems in their speech applications.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4881252Jun 15, 1988Nov 14, 1989Tunstall Telecom LimitedSheltered housing scheme communications systemUS6192343 *Dec 17, 1998Feb 20, 2001International Business Machines CorporationSpeech command input recognition system for interactive computer display with term weighting means used in interpreting potential commands from relevant speech termsUS6510411 *Oct 29, 1999Jan 21, 2003Unisys CorporationTask oriented dialog model and managerUS6895083May 2, 2001May 17, 2005Verizon Corporate Services Group Inc.System and method for maximum benefit routingUS7024368 *Feb 11, 2000Apr 4, 2006Microsoft CorporationMan-machine dialogue system, controls dialogue between system and user using dialogue specification employing augmented transition networks propagating tokensUS7026389Apr 9, 2001Apr 11, 2006Weyerhaeuser CompanyLabeling paint for use with metallic stencils on oriented strandboard finishing lineUS7062028Aug 4, 2003Jun 13, 2006Bellsouth Intellectual Property CorporationMethod and apparatus for routing calls based on the identification of the calling party or calling lineUS7092888Oct 24, 2002Aug 15, 2006Verizon Corporate Services Group Inc.Unsupervised training in natural language call routingUS7191133Feb 15, 2001Mar 13, 2007West CorporationScript compliance using speech recognitionUS7231019Feb 12, 2004Jun 12, 2007Microsoft CorporationAutomatic identification of telephone callers based on voice characteristicsUS7363212Mar 31, 2004Apr 22, 2008Avaya Technology Corp.Method and apparatus for translating a classification system into a target languageUS20020138265 *May 2, 2001Sep 26, 2002Daniell StevensError correction in speech recognitionUS20040068406 *Sep 27, 2002Apr 8, 2004Hidetsugu MaekawaDialogue apparatus, dialogue parent apparatus, dialogue child apparatus, dialogue control method, and dialogue control programWO2000055843A1 *Feb 11, 2000Sep 21, 2000Entropic LimitedMan-machine dialogue system and method* Cited by examinerNon-Patent CitationsReference1Banfield et al., J. D., "Model-Based Gaussian and Non-Gaussian Clustering", Biometrics, vol. 49, No. 3, pp. 803-821, Sep. 1993.2Basu et al., S., "Semi-supervised Clustering by Seeding", Proceedings of the 19th International Conference on Machine Learning (ICML-2002), Sydney, Australia, pp. 19-26, Jul. 2002.3Blum et al., A., "Combining Labeled and Unlabeled Data with Co-Training", The 11th Annual Conference on Computational Learning Theory, pp. 92-100, 1998.4Blum et al., A., "Learning from Labeled and Unlabeled Data using Graph Mincuts", Proceedings of the 18th International Conference on Machine Learning (ICML-2001), pp. 19-26, 2001.5Cadez et al., I., "Model-Based Clustering and Visualization of Navigation Patterns on a Web Site", Data Mining and Knowledge Discovery, vol. 7, pp. 399-424, 2003.6Castelli et al., V., "The Relative Value of Labeled and Unlabeled Samples in Pattern Recognition with an Unknown Mixing Parameter", IEEE Transactions on Information Theory, vol. 42, No. 6, pp. 2102-2117, Nov. 1996.7Cheeseman et al., P., "Bayesian Classification (AutoClass): Theory and Results", Advances in Knowledge Discovery and Data Mining, AAAI Press, Menlo Park, California, edited by U. M. Fayyad, G. Piatetslcy-Shapiro, P. Smyth and R. Uthurusamy, pp. 153-180, 1995.8Cohn et al., D., "Semi-supervised Clustering with User Feedback", Technical Report TR2003-1892, Cornell University, 2003.9Cozman et al., F. G., "Semi-Supervised Learning of Mixture Models", Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), Washington, D.C., pp. 106-113, 2003.10Dempster et al., A. P., "Maximum Likelihood from Incomplete Data via the EM Algorithm", Journal of the Royal Statistical Society, vol. 39, No. 1, 1977.11Fraley et al., C., "How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis", The Computer Journal, vol. 41, No. 8, pp. 578-588, 1998.12Guerrero-Curieses et al., A., "An Entropy Minimization Principle for Semi-Supervised Terrain Classification", Proceedings 2000 International Conference on Image Processing, vol. 3, No. 3, pp. 312-315, Sep. 2000.13Kamvar et al., S. D., "Interpreting and Extending Classical Agglomerative Clustering Algorithms using a Model-Based Approach", Proceedings of the Nineteenth International Conference on Machine Learning (ICML-2002), 2002.14Nigam et al., K., "Text Classification from Labeled and Unlabeled Documents using EM" Machine Learning, vol. 39, Nos. 2/3, pp. 103-134, May/Jun. 2000.15Smyth, P., "Probabilistic Model-Based Clustering of Multivariate and Sequential Data", In Proceedings of Seventh International Workshop on Artificial Intelligence and Statistics, San Francisco, California, pp. 299-304, 1999.16Wagstaff et al., K., "Constrained K-means Clustering with Background Knowledge", Proceedings of the Eighteenth International Conferences on Machine Learning (ICML-2001), pp. 577-584, 2001.17Zeng et al., H., "CBC: Clustering Based Text Classification Requiring Minimal Labeled Data", Proceedings of IEEE International Conference on Data Mining, Melbourne, Florida, 2003.18Zhong, S., "Probabilistic Model-based Clustering of Complex Data", doctoral dissertation, Department of Electrical and Computer Engineering, The University of Texas at Austin, Aug. 2003.19Zhong, S., "Semi-Supervised Model-based Document Clustering: A Comparative Study", Department of Computer Science and Engineering, Florida Atlantic University, 2004.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8099279Feb 9, 2005Jan 17, 2012Microsoft CorporationInteractive clustering method for identifying problems in speech applicationsUS20070168190 *Oct 30, 2006Jul 19, 2007Konica Minolta Business Technologies, Inc.Information processing apparatus with speech recognition capability, and speech command executing program and method executed in information processing apparatus* Cited by examinerClassifications U.S. Classification704/251, 704/246, 704/254, 704/9International ClassificationG10L15/00Cooperative ClassificationG10L15/083, G10L15/01European ClassificationG10L15/01, G10L15/08NLegal EventsDateCodeEventDescriptionMar 18, 2013FPAYFee paymentYear of fee payment: 4Mar 29, 2005ASAssignmentOwner name: MICROSOFT CORPORATION, WASHINGTONFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ACERO, ALEJANDRO;YU, DONG;REEL/FRAME:015833/0462Effective date: 20050207RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google

type: global

value: This website is a search engine provided by Google. Handling over 3.5 billion searches per day, it has a 92% share of the global search engine market.


text: This book was agony. I was so looking forward to it. I read Longitude and loved it. I couldn't listen to this at all. I only got about 1/3 of the way through before I gave up. It was all fluff and not much science. Nothing I found interesting. 1 of 1 people found this review helpful

value: This website is an American online audiobook and podcast service that allows users to purchase and stream audiobooks and other forms of spoken word content. This content can be purchased individually or under a subscription model where the user receives "credits" that can be redeemed for content monthly and receive access to a curated on-demand library of content.


text:
Monica Robles on Etsy
45 Following 114 Followers
Welcome to Paper & Plarn!
My name is Monica, I run this shop and am very excited that you decided to visit it. I'm an artist with many odd jobs. I like to play with plastic and paper because they are so abundant in my life. Every time I create something out of these (and other) materials it f …eels like I'm bringing something to life.
Below are the guidelines I pledged to follow for my shop:
Handmade Design: One of a kind items are handcrafted by an independent artist with care, providing alternatives to mass-produced products.
Sustainability through Materials: I gather post-consumer materials through methods that avoid overuse of resources, promote reduction of industrial waste, and educate the community on the characteristics of the materials. Other newly sourced materials (such as paper raffia for gift wrapping) have been selected for giving the design its eco-friendly characteristics.
Local travel: The material used in the products travels locally reducing the polluting consequences of transportation.
Reduced Packaging: I use post-consumer, handmade, up-cycled or recycled packaging and labeling. Each item is properly labeled to promote the reuse and proper disposal of the packaging.
Thanks again for visiting again! And don't forget to look at my shop and blog at:
www.etsy.com/shop/PAPERandPLARN
urbanresources.blogspot.com
Pink Cherry Blossom Brooch - Vintage Flower Stem Brooch with Bright Pink Crystal Accents - Eco Mother's Day
Peony Blossom vintage pajamas
SALE - Summer Moon - Genuine Sea Glass Jewelry - Bezel Set Pendant
Crocheted Statement Necklace RESERVED FOR DRSPHINX - Coral Ruffle on Silver Chain
Beeswax Candles White Candle Taper Wedding Candles Gift Set 6 Thin Japanese Candles Zen Candles Pure Beeswax Rolled Candles Asian Inspired
Ready to Ship for Mother's Day. Alfred Shaheen Vintage Handprint in Tiare Tapa-Green. Repurposed Burlap Coffee Bag Hobo Handbag and Purse.
Vintage Button Brooch Applejuice Bakelite Vintage Button Jewelry
Plain Kraft Pillow Boxes, 3.5x3x1 - Wholesale, Pack of 300
Reserved for SouthernDuo - Fabric art brooch pin - Turquoise Brown Autumn Leaf
Scarlet, lime green and chocolate brown vintage linen belt
cloth doll, fish on a string, 2012 version
Hanging Soap Caddy Bag Orange Blue Recycled Billboard
Reversible citrus orange and apricot market day tote HALF PRICE
Choose your Rebound Journal // Elementary School Books by PrairiePeasant
Floral folk dress revamped vintage ethnic red small FOLK FEMME
LindenAndLime

value: This website is an American e-commerce company focused on handmade or vintage items and craft supplies.


text:
Amazon.com: World Pride 50's 60's Grease Cateye Glasses Rhinestone for Fancy Dress (Black): Adult Sized Costumes: Clothing
World Pride® 50's 60's Grease Cateye Glasses Rhinestone for Fancy Dress
$5.99 $5.99 $5.99 $5.99 In Stock.
Ships from and sold by Yaheetech.
Newly made with a vintage 50s look
A fun accessory for 50's parties, plays or dances
$3.88 - $6.88
Sold by Yaheetech
Hip Hop 50s Shop Cateye Glasses Child/Youth - Black
$7.99 Prime 50's Chiffon Scarf 1950s Grease Costume Scarf 1616
Hip Hop 50s Shop Womens Bobby Sock W/Poodle Applique
Hey Viv ! Women's Sheer Chiffon Scarf
$28.99 FunWorld Poodle Skirt Costume
Approx Dimensions: Lens Size: 5.4x3.5 cm/ 2.1"x1.4" (L*W); Arm Length: 14cm / 5.5"; Bridge Width: 1.3cm / 0.5"; Frame Width: 15.5cm / 6.1"
B00FO1DSFY
Amazon Best Sellers Rank: #7,551 in Clothing (See Top 100 in Clothing) Average Customer Review:
3 of 3 people found the following review helpfulI get Compliments all the time
on May 14, 2013Color: Black
Everyone loves my cateye glasses. I get compliments on them at least 1 a day.thank you so much, i want to order all colors. I dress rockabilly and pin up and they match perfect .
3 of 3 people found the following review helpfulSoo adorable! <3
Angel Demure
on May 18, 2013Color: Black
I have wanted these 50s cateye glasses for a while now to actually wear with everyday outfits. These glasses are the exact pair that I found at a retail store for $6.99. The quality is great, its durable, and very cute! Very fast shipping too!
1 of 1 people found the following review helpfulDidn't Last Long
Jennifer U. Heston
on October 31, 2013Color: Black
These glasses look great - I loved them when I got them and I was so excited to wear them with my 1950's costume for Halloween. Well, I got to wear them for about an hour on Halloween before they broke. The hinge connecting the front of the glasses to the bows are made of plastic. As manufactured, the hinge was a bit stiff, but not terribly so. After closing and opening the glasses only a few times, the hinge broke apart and the bow fell off. I am going to try to glue it in place to see if it will hold any longer, but I am not holding out much hope. These looked fantastic while they lasted and really made my 1950s costume come alive. I only wish that they lasted longer.
1 of 1 people found the following review helpfulGood fit, minor inaccuracy
on January 20, 2015Color: White
The glasses are almost exactly as pictured. Almost. The rhinestones were arranged in a linear triangle pattern rather than the pictured pattern, which disappointed me, as the pictured shape would have been perfect for the character I was dressed up as. Otherwise, I am satisfied with my purchase and look forward to wearing these glasses when I dress up as the character.
1 of 1 people found the following review helpfulFancy
on December 16, 2013Color: Pink
My 6year old granddaughter loves these. Her best friend wears glasses and she will wear these in support of her friend. She also plays dress up with them. They are so cute on her and very well made.
3 of 4 people found the following review helpfulreaders with a flair
Joyce E. Wardlaw
on June 18, 2013Color: Black
Yes, with a flair. With my 60's dresses (very flare and below the knee) they work out just fine. I couldn't be more happy. I love them immensely. Makes for a great looking reader and add the chain...what can I say.
on May 12, 2015Color: Black
Ordered for my 50's housewife bridal shower and they were a huge hit! Great buy for the price!Re another reviewers comment that they broke after opening and closing them a few times, buyers should note that they did not for me. I purchased these in 2013 and they're still working fine for me and both my sister and myself have had several opportunities to wear them. They're not hundred dollar designer frames, but are a great buy for the price.
on November 14, 2014Color: Black
These were the perfect addition to the50s outfit. Perfect.
Very light wieght.
Published 1 day ago by Melinda Gehrking
Good quality & fit for price
Published 29 days ago by Tonya
Published 1 month ago by Alysha N.
Thought this was for a grown up but its for a kid
Published 1 month ago by fabulousc02
I thought the would fit a child but they are big for affluent size
Published 2 months ago by john downs
I WILL use these down the road. They ARE ...
I WILL use these down the road. They ARE everything that they are described as, but, "I" was thinking sunglasses when I ordered them...
Published 2 months ago by 1Princess
Exactly how pictured. We were pleased with it
Published 3 months ago by Stacy Neal
and they fit perfect.
Bought for 8- and 6-year-old girls, and they fit perfect.
Published 3 months ago by YMD ONE
Great prop for 50s party
Glasses are a good quality and fit well with a reasonable price.
Published 3 months ago by RC
Not bad for the price. Perfect for the 50s party I will be attending. If you wear a lot of makeup may want to choose another color frame.
Published 3 months ago by Tammy Trueblood
Hip Hop 50s Shop Adult Cateye Rhinestone Glasses
$7.99 Prime Rhinestone Cat Eye 50s Party Glasses in Many Colors
$2.08 - $9.99
50's Chiffon Scarf 1950s Grease Costume Scarf 1616
$6.33 Prime Hip Hop 50s Shop Womens Bobby Sock W/Poodle Applique
Check out these pages to see more: 50s costume, women dress with cat, eyes glasses vintage women, cat woman costume accessories, eye glasses black woman, vintage cat glasses

value: This website is an American multinational technology company which focuses on e-commerce, cloud computing, digital streaming, and artificial intelligence.


text:
8686 Various Schools of Thought | God | Grace In Christianity
B.D. 8686 27.11.1963 Various schools of thought.... Pretended worship....
It is just like the time when Christ descended to earth…. People’s thoughts are just as confused, they advocate the same misguided teachings, they believe to possess the sole truth and yet they are living right in the midst of falsehood…. Now, as then, they uphold their hollow creeds even though all vary from each other, each creed endorses its own point of view and each one is convinced that it can lay claim to the truthfulness of its teachings. And the number of schools of thought has grown considerably, there are not just a few opposing each other but a great variety of opinions has created a great variety of faith communities which all endeavour to win supporters and fanatically insist on the doctrines they endorse…. Hence it is essential that people receive the truth, that they at last learn what God expects from people…. It is essential that God’s will is revealed to them so that they learn to recognise that the fulfilment of His will is the meaning and purpose of life on earth…. However, every church or spiritual movement teaches that it proclaims God’s will and still the various creeds differ. At the time of Christ’s descent to earth people prided themselves just like today on practicing various customs, the divine service was fanatically performed and lacked spirit so that people did not establish any heartfelt contact with their God and Creator…. And the same happens nowadays: a formal religious service giving due consideration to all rites and customs, demonstrating to the world the union with God whilst the heart of the individual is far removed from this heartfelt bond with Him…. It is just pretence and deception, there is no longer any truthfulness in people’s way of life.... They solely belong to the physical world, and even if they want to give the impression of a relationship with God they only perform lifeless formalities which do not touch their souls. And you, who receive the truth from God, shall challenge this pretence of worship and remember that even Jesus during His life on earth did not support those who upheld wrong doctrines and tried to justify their pretence of worship as the only one pleasing to God and who condemned everything that contradicted their teachings. Nowadays people will not want to part with the doctrines handed down to them by people either and will therefore always remain in error, because they refuse to accept the pure truth which can only be granted to them from above…. from the Eternal Truth. Yet such a gift of grace requires free voluntary acceptance.... The truth cannot be forcibly conveyed to people who are unwilling to accept it. And therefore it is difficult to root out misguided teachings even if these are substantiated and amended to them as being wrong. There will only ever be a few who will be open-minded to what God Himself conveys to people, because they don’t want to acknowledge such evident working of God…. because they have no knowledge of the fact that God pours out His spirit over those who willingly open up their hearts to Him for the receipt of His pure truth…. Prior to Jesus’ descent to earth, before His crucifixion, the pouring out of the spirit upon a human being was not possible, and therefore the world was full of error…. However, after His death on the cross truth could find access to mankind and it became possible to fully enlighten people as to whether and when their thinking was erroneous, it was possible to instruct them through the spirit and they could rid themselves of all error. Yet there was never any coercion and this is why time and again error could creep in and be advocated as truth and passed on with such tenacity that it is difficult to convince people of their wrong thinking, and thus they will always resist accepting the pure truth from God and giving up their misconceptions. And still, for the sake of truth itself everything has to be tried to make people change their points of view.... everything has to be tried to motivate them to think, everything has to be done to impart the pure truth to them, for they can only reach the final goal on the path of truth…. As long as they remain in their wrong thinking they will not be able to partake in the bliss of the kingdom of the beyond, which can only ever guarantee enlightenment…. Only truth grants light, therefore truth alone leads to beatitude, for countless beings can be made happy with it again and happiness in the spiritual kingdom consists of the giving of light. This is why you humans must not fall victim to wrong thinking, this is why the truth is offered to you time and again, this is why everything will be done on the part of God to turn your thinking in the right direction, yet you yourselves need only have the will to live in truth…. For it is not as important on earth as it is in the kingdom of the beyond that you know the truth, because you cannot be happy there without truth, and even if you ignore your dark state on earth…. it will nevertheless torment you in the beyond, because only light is beatitude and light is only ever emanated by the Truth, by God Himself, Who is the eternal Essence of light…. Amen
Documents Similar To 8686 Various Schools of Thought
95281636-The-Laws-of-the-Spirit-Bob-Buess.pdf

value: This website is an American e-book and audiobook subscription service that includes one million titles.


text:
Details about Ultra Thin Crystal Clear Snap-on Hard Case Cover for iPhone 4 G 4S 4GS
Ultra Thin Crystal Clear Snap-on Hard Case Cover for iPhone 4 G 4S 4GS
16:56:59 PDT
emallbox
Card Pocket ShockProof Slim...
For iPhone 4 4S Aluminum Chrome...
Ultra-Thin Slim Hard Back Rubber Shell...
eBay item number:110920244048
The Best iPhone 6 Cases,
Clear Case/Cover For LG Optimus G,
Clear Bumper Case For iPhone 4s,
Case/Cover For iPhone 4s,
Clear Fitted Case For iPhone 4s,
Clear Case For iPhone 4

value: This website is an American multinational e-commerce corporation based in San Jose, California, that facilitates consumer-to-consumer and business-to-consumer sales through its website.


text:
Recoil Reducer Shooting Rest Adjustable Sturdy Vice Target Steel With Sandbags | eBay
See more Recoil Reducer Gun Rest Rifle Range Adjustable...
GUIDE GEAR Recoil Reducer Shooting Rest Adjustable Steel Frame Ballast Sand Bag
Portable Rifle Target Shooting Arm Rest Recoil Reducing Steel Frame Adjustable
odysseus-artifacts (136 )
Details about Recoil Reducer Shooting Rest Adjustable Sturdy Vice Target Steel With Sandbags
200 Bulk Pack Silhouette hand gun, rifle paper shooting targets 11X17
312641899829
Last updated on Sep 17, 2019 17:26:08 PDT View all revisions
Guide Gear 10301001
Type: Shooting Rest Brand:
by faulkit2 Sep 22, 2019
Great recoil reduction. Excellent for sighting in your rifle before hunting season. fill the nylon bag with sand and you are good to go.
Back to home page | See More Details about "Recoil Reducer Gun Rest Rifle Range Adjustable Sturdy ..." Return to top
Ar500 Steel Shooting Targets Indiana Range & Shooting Targets,


text:
Molecular basis of invasion in breast cancer. - PubMed - NCBI
Format: AbstractFormatSummarySummary (text)AbstractAbstract (text)MEDLINEXMLPMID ListApplySend toChoose DestinationFileClipboardCollectionsE-mailOrderMy BibliographyCitation managerFormatSummary (text)Abstract (text)MEDLINEXMLPMID ListCSVCreate File1 selected item: 17957337FormatSummarySummary (text)AbstractAbstract (text)MEDLINEXMLPMID ListMeSH and Other DataE-mailSubjectAdditional textE-mailDidn't get the message? Find out why...Add to ClipboardAdd to CollectionsOrder articlesAdd to My BibliographyGenerate a file for use with external citation management software.Create File
See comment in PubMed Commons belowCell Mol Life Sci. 2007 Dec;64(24):3201-18.Molecular basis of invasion in breast cancer.McSherry EA1, Donatello S, Hopkins AM, McDonnell S.Author information1UCD School of Biomolecular and Biomedical Science, University College Dublin, Belfield, Dublin 4, Ireland.AbstractCancer cell invasion involves the breaching of tissue barriers by cancer cells, and the subsequent infiltration of these cells throughout the surrounding tissue. In breast cancer, invasion at the molecular level requires the coordinated efforts of numerous processes within the cancer cell and its surroundings. Accumulation of genetic changes which impair the regulation of cell growth and death is generally accepted to initiate cancer. Loss of cell-adhesion molecules, resulting in a loss in tissue architecture, in parallel with matrix remodelling may also confer a motile or migratory advantage to breast cancer cells. The tumour microenvironment may further influence the behaviour of these cancer cells through expression of cytokines, growth factors, and proteases promoting chemotaxis and invasion. This review will attempt to summarise recent work on these fundamental processes influencing or facilitating breast cancer cell invasion. (Part of a Multi-author Review).PMID: 17957337 DOI: 10.1007/s00018-007-7388-0 [PubMed - indexed for MEDLINE] SharePublication Types, MeSH Terms, SubstancesPublication TypesReviewMeSH TermsBreast Neoplasms/classificationBreast Neoplasms/diagnosisBreast Neoplasms/genetics*Breast Neoplasms/pathology*Cell Adhesion/physiologyCell Movement/physiologyExtracellular Matrix/physiologyGene Expression ProfilingGene Expression Regulation, NeoplasticHumansMetalloproteases/physiologyModels, BiologicalNeoplasm InvasivenessNeoplasm MetastasisPrognosisTumor Escape/physiologySubstancesMetalloproteasesLinkOut - more resourcesFull Text SourcesSpringerOther Literature SourcesCOS Scholar UniverseMedicalBreast Cancer - Genetic AllianceBreast Cancer - MedlinePlus Health InformationPubMed Commons home

value: This website is part of the United States National Library of Medicine , a branch of the National Institutes of Health . It is approved and funded by the government of the United States.


text:
See more: need build website weight loss planner, help build website, build website css div, website builder, best website builder, how much does it cost to build a website for a small business, wix, how much does it cost to hire someone to build a website, wix website, squarespace, i need a website designer, build website construction business, adult website consulting services, build website share advertising revenue, hr staffing consulting services website, website design consulting services, build website home remodelling services, build website registration login user pages, website 8 pages, how to build a advertising website free advice
( 3 reviews ) Garching, Germany
Project ID: #19263259
Hello!!! I am happy to get this opportunity to bid your project. I read your requirement carefully. I can make a beautiful site using proper builder and editer such as elementor, gutenburg, visual composer(WPBak More
210 freelancers are bidding on average $416 for this job
Hello, Greeting from Votive Technologies!! As per for project requirement, we can surely design and develop website for your consulting services with required pages. We have completed 750+ projects on various More
It appears that you want to create a consulting website for 8 pages in total. I think it will be nice if you can share the pages/sections which are required. I want to know if the contents are static, or there are any More
Hello, we are the team of highly-qualified professionals, who have a lot of experience in creating and designing websites. We can create a website with 8 pages for your business. Please give us more details what would More
Hi, We are team of professional designers and developers with 7+ years of experience and interested in your project, give you professional design layout for advertising consulting services. Website will be respon More
Hello CHR, We can design and develop Advertising Website for your consulting Service within timeframe with your permission. Please have a look at some of our past work experiences which are big portals: - Homm More
Hello Client! We have understand that you want to Build a website for advertising my consulting services. There are total 8 pages. Please share required images and contents . List of functionalities you want More
Hi there! I will design an attractive & modern WordPress website for your business! I'm well experienced in designing stunning websites which support you to achieve your targets. Please see below portfolio links for More
Hi, I can design your Advertising Consulting Website with 8 pages as per your requirements, I assure you that this Website will responsive to all devices and I assure quality work and timely support. Q. Do you have re More
Hello, Please come to chat so that i can gather details for these 8 pages layout. Portfolio - https://www.freelancer.com/u/mciittech.html CSS HTML PHP Wordpress SEO	Please come to chat so that we will discuss i More
i can provide SEO optimized, mobile RESPONSIVE website for your CONSULTING SERVICES business in your budget. the website will be traffic driven and graphically loaded with fast loading and easy to manage and maintai More
Hi There, I hope you are doing good. I can work on your html project. Please confirm the whole scope of work. It would be great if you allow me to discuss on chat box. I have been done more than 500 projects More
Serverless AWS (€12-18 EUR / hour)

value: This website is an Australian freelance marketplace website, which allows potential employers to post jobs that freelancers can then bid to complete. Founded in 2009, its headquarters is located in Sydney, Australia, though it also has offices in Vancouver, London, Buenos Aires, Manila, and Jakarta.


text:
Boutique Hotels in Chelsea : Find 3119 Hotel Deals | Expedia
Featured Chelsea Boutique Hotels
See all 3,119 Hotels in Chelsea
Wonderful location in a safe and clean neighborhood which is also close to everything. The staff makes this hotel because they are top notch. Would definitely stay here again BUT only with a balcony room.
Oasis of quiet and comfort in London. Great staff make this hotel really special. Excellent location in a quiet neighborhood 5 minutes walk from the underground. Wonderful to come "home" to this haven after touring in London and surrounding areas.
Simply the best in all respects. It's like having your own private flat in one of the best and most peaceful parts of London. William, Johnny, Richie, and all of the staff we interacted with were wonderful. Truly 5 star accommodations.
Lovely big clean rooms and the hotel location is nice and quiet but within walking distance of South Kensington tube.
Back Again! The Rembrandt is the real deal. I have stayed at several hotels in London but I think I found a jewel. It is close to museums, restaurants and the Piccadilly underground line which takes you out to Heathrow. Throw in free wifi and breakfast--who can beat this hotel!
good hotel but food needs to be improve nice experience,very good good exellent good good good good good good good
Nice hotel It is a very clean and good location but it is noisy and busy at night
Nice Neighborhood I chose this hotel because it was close to a good tube stop in a safe neighborhood and near some sights I wanted to see. It served those purposes but I woud not stay here again because it is overpriced.
Great location Close to tube station and central London, decent sized room with kitchen and washing machine!
Recently booked Boutique Hotels in Chelsea
Reserve a Chelsea Boutique Accommodation that is Uniquely You
Once you've arrived in Chelsea and realize that you've left all your worries behind, you can relax for the next several days. Breath deeply once you walk out of the airport and inhale the incredible feeling of being able to go wherever you want to go. Wherever you might holiday, you still bring your one and only self on your trip, and Expedia provides great rates at the most distinctive boutique properties in the city.
You'll love coming back from a fun-filled day of sightseeing in Chelsea and relaxing in a space that boasts intricate designs and fabulous murals. Moreover, these distinctive accommodations bring a bit of the culture and vibe of Chelsea into your room, making your stay that much more intimate. Almost all boutique hotels have qualities like:
A prime location in the center of Chelsea. You'll be moments away from the area's best sites.
An idiosyncratic character that you can't be able to find in grander hotels, from individually crafted rooms to intimate bars, and creature comforts that make them feel homey.
Bask in Everything Chelsea Offers!
Chelsea has accommodations to appeal to all types of budgets and preferences that will create a stay unforgettable.
Book Some Design-Driven Digs in Chelsea
From hotels that weave modern touches with classic styling to ones that place you in a completely new world, you are sure to geta lodging that is completely you. Your Chelsea boutique hotel is only a couple taps away. Expedia makes it easy to select your preferable place to stay when you visit Chelsea.
Epsom Hotels (11.95 mi)
Chelsea Boutique Hotels & More
Boutique Hotels Near Popular Chelsea Attractions
Chelsea Boutique Hotels by Star Rating
Shimla Lodging
Rental Car Melilla

value: This website is an online travel agency owned by Expedia Group, an American online travel shopping company based in Seattle.


text:
Toby Jug by Denis O'Connor (Paperback, 2014) | eBay
Toby Jug by Denis O'Connor (Paperback, 2014)
Toby Jug by Denis O'Connor 9781472111753 (Paperback, 2014) Delivery UK delivery is usually within 8 to 10 working days. International delivery varies by country, please see the Wordery store help page for details.
item 1 Toby Jug By Denis O'Connor -Toby Jug By Denis O'Connor
item 2 Toby Jug,Denis O'Connor -Toby Jug,Denis O'Connor
item 3 Toby Jug: a year in the life of a rescued cat by Denis O'Connor (Paperback) -Toby Jug: a year in the life of a rescued cat by Denis O'Connor (Paperback)
item 4 Toby Jug-ExLibrary -Toby Jug-ExLibrary
item 5 Toby Jug by O'Connor, Denis Book The Cheap Fast Free Post -Toby Jug by O'Connor, Denis Book The Cheap Fast Free Post
item 6 Toby Jug by O'Connor, Denis Book The Cheap Fast Free Post -Toby Jug by O'Connor, Denis Book The Cheap Fast Free Post
item 7 Toby Jug by Denis O'Connor 9781472111753 (Paperback, 2014) -Toby Jug by Denis O'Connor 9781472111753 (Paperback, 2014)
Moone Boy: The Blunder Years by Chris O'Dowd (Hardback, 2015)
Denis O'Connor trained as a psychologist and teacher. Throughout his career he taught in schools and lectured in colleges and universities. He holds a doctorate in education and psychology. He is retired and lives with his wife Catherine and his two Maine Coone cats on the South coast
1472111753
9781472111753
209531840
by jeanjeanieq08 Feb, 2017
An enjoyable book which I enjoyed very much
by sue*suzy119 Jan, 2017
Having read all his previous books I was not disappointed. I wish Toby jug was mine!
by littlemil6510 May, 2016
lovely story, about a man who loves his cats! and a loving cat too, just don't want to put down.